[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:59:10 dist-test-slave-2lzq krb5kdc[27114](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:59:10 dist-test-slave-2lzq krb5kdc[27114](info): set up 2 sockets
Sep 29 10:59:10 dist-test-slave-2lzq krb5kdc[27114](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260929 10:59:11.034170 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 10:59:11 dist-test-slave-2lzq krb5kdc[27114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:59:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:59:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.120.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
I20260929 10:59:11.152838 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:35437
--webserver_interface=127.26.120.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:37553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:35437
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:11.235163 27127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:11.235374 27127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:11.235397 27127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:11.236444 27127 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:59:11.236478 27127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:11.236493 27127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:11.236505 27127 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:59:11.236517 27127 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:59:11.237933 27127 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:37553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:35437
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:35437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:11.238260 27127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:11.238492 27127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 10:59:11.243667 27127 server_base.cc:1034] running on GCE node
W20260929 10:59:11.243603 27135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:11.243594 27133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:11.243595 27132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:11.244177 27127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:11.244580 27127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:11.245759 27127 hybrid_clock.cc:648] HybridClock initialized: now 1790679551245725 us; error 44 us; skew 500 ppm
Sep 29 10:59:11 dist-test-slave-2lzq krb5kdc[27114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679551, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:11.248505 27127 init.cc:377] Logged in from keytab as kudu/127.26.120.254@KRBTEST.COM (short username kudu)
I20260929 10:59:11.248977 27127 webserver.cc:460] Webserver started at http://127.26.120.254:34847/ using document root <none> and password file <none>
I20260929 10:59:11.249228 27127 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:11.249301 27127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:11.249491 27127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:11.250905 27127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "ae4096e2e76747ba8c468e179c07220e"
format_stamp: "Formatted at 2026-09-29 10:59:11 on dist-test-slave-2lzq"
server_key: "fcb841f0908056b7ba7d8b9a5069f7d5"
server_key_iv: "1c80bc6390a8d15c817c1d51119b2d9a"
server_key_version: "encryptionkey@0"
I20260929 10:59:11.251338 27127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "ae4096e2e76747ba8c468e179c07220e"
format_stamp: "Formatted at 2026-09-29 10:59:11 on dist-test-slave-2lzq"
server_key: "fcb841f0908056b7ba7d8b9a5069f7d5"
server_key_iv: "1c80bc6390a8d15c817c1d51119b2d9a"
server_key_version: "encryptionkey@0"
I20260929 10:59:11.252990 27127 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 10:59:11.253844 27142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:11.254089 27127 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:59:11.254206 27127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "ae4096e2e76747ba8c468e179c07220e"
format_stamp: "Formatted at 2026-09-29 10:59:11 on dist-test-slave-2lzq"
server_key: "fcb841f0908056b7ba7d8b9a5069f7d5"
server_key_iv: "1c80bc6390a8d15c817c1d51119b2d9a"
server_key_version: "encryptionkey@0"
I20260929 10:59:11.254293 27127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:11.267334 27127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:11.269562 27127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:11.269733 27127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:11.274519 27127 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:35437
I20260929 10:59:11.274627 27194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:35437 every 8 connection(s)
I20260929 10:59:11.275029 27127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 10:59:11.275945 27195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:11.277781 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 27127
I20260929 10:59:11.277897 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 10:59:11.278693 27107 external_mini_cluster.cc:1428] Setting key d6926bdabaaa7c9d9057a1b07a43ddff
I20260929 10:59:11.278963 27195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae4096e2e76747ba8c468e179c07220e: Bootstrap starting.
I20260929 10:59:11.279789 27195 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae4096e2e76747ba8c468e179c07220e: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:11.280354 27195 log.cc:824] T 00000000000000000000000000000000 P ae4096e2e76747ba8c468e179c07220e: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:11.281316 27195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae4096e2e76747ba8c468e179c07220e: No bootstrap required, opened a new log
I20260929 10:59:11.283479 27195 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae4096e2e76747ba8c468e179c07220e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4096e2e76747ba8c468e179c07220e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 35437 } }
I20260929 10:59:11.283641 27195 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae4096e2e76747ba8c468e179c07220e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:11.283687 27195 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae4096e2e76747ba8c468e179c07220e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae4096e2e76747ba8c468e179c07220e, State: Initialized, Role: FOLLOWER
I20260929 10:59:11.283854 27195 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae4096e2e76747ba8c468e179c07220e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4096e2e76747ba8c468e179c07220e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 35437 } }
I20260929 10:59:11.283946 27195 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae4096e2e76747ba8c468e179c07220e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:59:11.283978 27195 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae4096e2e76747ba8c468e179c07220e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:59:11.284018 27195 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae4096e2e76747ba8c468e179c07220e [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:11.285082 27195 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae4096e2e76747ba8c468e179c07220e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4096e2e76747ba8c468e179c07220e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 35437 } }
I20260929 10:59:11.285202 27195 leader_election.cc:302] T 00000000000000000000000000000000 P ae4096e2e76747ba8c468e179c07220e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae4096e2e76747ba8c468e179c07220e; no voters:
I20260929 10:59:11.285409 27195 leader_election.cc:288] T 00000000000000000000000000000000 P ae4096e2e76747ba8c468e179c07220e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:59:11.285503 27200 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae4096e2e76747ba8c468e179c07220e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:59:11.285671 27200 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae4096e2e76747ba8c468e179c07220e [term 1 LEADER]: Becoming Leader. State: Replica: ae4096e2e76747ba8c468e179c07220e, State: Running, Role: LEADER
I20260929 10:59:11.285701 27195 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae4096e2e76747ba8c468e179c07220e [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:59:11.285770 27200 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae4096e2e76747ba8c468e179c07220e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4096e2e76747ba8c468e179c07220e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 35437 } }
I20260929 10:59:11.286206 27202 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae4096e2e76747ba8c468e179c07220e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae4096e2e76747ba8c468e179c07220e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4096e2e76747ba8c468e179c07220e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 35437 } } }
I20260929 10:59:11.286307 27202 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae4096e2e76747ba8c468e179c07220e [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:11.286195 27200 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae4096e2e76747ba8c468e179c07220e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae4096e2e76747ba8c468e179c07220e. Latest consensus state: current_term: 1 leader_uuid: "ae4096e2e76747ba8c468e179c07220e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4096e2e76747ba8c468e179c07220e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 35437 } } }
I20260929 10:59:11.286490 27200 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae4096e2e76747ba8c468e179c07220e [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:11.286763 27209 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:59:11.287720 27209 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:59:11.289507 27209 catalog_manager.cc:1347] Generated new cluster ID: c8b22e1f7eb544538b9a55ff2e9b9649
I20260929 10:59:11.289723 27209 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 10:59:11 dist-test-slave-2lzq krb5kdc[27114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:11.297159 27198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:11.279709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51606 (local address 127.26.120.254:35437)
0929 10:59:11.280053 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:11.280062 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:11.280228 (+ 166us) server_negotiation.cc:407] Connection header received
0929 10:59:11.283458 (+ 3230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:11.283464 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:11.283541 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:11.283685 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:11.285963 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:11.287507 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:11.288902 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:11.289444 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:11.289572 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:11.293039 (+ 3467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:11.293045 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:11.293049 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:11.293059 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:11.295191 (+ 2132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:11.295665 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:11.295668 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:11.295668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:11.295768 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:11.296036 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:11.296037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:11.296037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:11.296259 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:11.296432 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:11.296714 (+ 282us) server_negotiation.cc:299] Negotiation successful
0929 10:59:11.296858 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":142,"threads_started":1}
I20260929 10:59:11.307271 27209 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:59:11.307816 27209 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:59:11.312101 27209 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae4096e2e76747ba8c468e179c07220e: Generated new TSK 0
I20260929 10:59:11.312289 27209 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.120.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab.
Entry for principal kudu/127.26.120.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab.
I20260929 10:59:11.343187 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.193:0
--local_ip_for_outbound_sockets=127.26.120.193
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:35437
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:37553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:11.426680 27221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:11.426903 27221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:11.426927 27221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:11.428148 27221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:11.428190 27221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:11.428217 27221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.193
I20260929 10:59:11.429677 27221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:37553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:35437
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.193
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:11.429977 27221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:11.430240 27221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:11.435515 27226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:11.435532 27227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:11.435529 27229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:11.435673 27221 server_base.cc:1034] running on GCE node
I20260929 10:59:11.436039 27221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:11.436290 27221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:11.437474 27221 hybrid_clock.cc:648] HybridClock initialized: now 1790679551437452 us; error 30 us; skew 500 ppm
Sep 29 10:59:11 dist-test-slave-2lzq krb5kdc[27114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679551, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:11.440061 27221 init.cc:377] Logged in from keytab as kudu/127.26.120.193@KRBTEST.COM (short username kudu)
I20260929 10:59:11.440560 27221 webserver.cc:460] Webserver started at http://127.26.120.193:43343/ using document root <none> and password file <none>
I20260929 10:59:11.440801 27221 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:11.440871 27221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:11.441001 27221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:11.442099 27221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-0/data/instance:
uuid: "cad95d67753e4b0c889e9b8ef5e4098c"
format_stamp: "Formatted at 2026-09-29 10:59:11 on dist-test-slave-2lzq"
server_key: "c3885363cc3397edc48655fc9f44f056"
server_key_iv: "d2a701f9989650940840bb7765bd27c3"
server_key_version: "encryptionkey@0"
I20260929 10:59:11.442476 27221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance:
uuid: "cad95d67753e4b0c889e9b8ef5e4098c"
format_stamp: "Formatted at 2026-09-29 10:59:11 on dist-test-slave-2lzq"
server_key: "c3885363cc3397edc48655fc9f44f056"
server_key_iv: "d2a701f9989650940840bb7765bd27c3"
server_key_version: "encryptionkey@0"
I20260929 10:59:11.443774 27221 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:59:11.444532 27236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:11.444940 27221 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:59:11.445065 27221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-0/wal
uuid: "cad95d67753e4b0c889e9b8ef5e4098c"
format_stamp: "Formatted at 2026-09-29 10:59:11 on dist-test-slave-2lzq"
server_key: "c3885363cc3397edc48655fc9f44f056"
server_key_iv: "d2a701f9989650940840bb7765bd27c3"
server_key_version: "encryptionkey@0"
I20260929 10:59:11.445159 27221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:11.464003 27221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:11.465973 27221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:11.466163 27221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:11.466405 27221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:59:11.466756 27221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:11.466812 27221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:11.466854 27221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:11.466885 27221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:11.473799 27221 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.193:36163
I20260929 10:59:11.473843 27349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.193:36163 every 8 connection(s)
I20260929 10:59:11.474267 27221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
I20260929 10:59:11.478030 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 27221
I20260929 10:59:11.478168 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance
I20260929 10:59:11.478322 27107 external_mini_cluster.cc:1428] Setting key e9a27949e619bdc7eeac7fd6b56eda7c
Sep 29 10:59:11 dist-test-slave-2lzq krb5kdc[27114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679551, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:11.484619 27352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:11.475201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:35437 (local address 127.26.120.193:46543)
0929 10:59:11.475383 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:11.475426 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:11.475888 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:11.476105 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:11.476109 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:11.476688 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:11.476886 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:11.476891 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:11.477572 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:11.477572 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:11.478584 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:11.478588 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:11.479096 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:11.479097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:11.479256 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:11.479747 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:11.479758 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:11.481359 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:11.483256 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:11.483262 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:11.483264 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:11.483480 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:11.483703 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:11.483704 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:11.483704 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:11.483801 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:11.484113 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:11.484118 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:11.484237 (+ 119us) client_negotiation.cc:769] Sending connection context
0929 10:59:11.484328 (+ 91us) client_negotiation.cc:240] Negotiation successful
0929 10:59:11.484445 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":87,"threads_started":1}
I20260929 10:59:11.484619 27198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:11.475221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:46543 (local address 127.26.120.254:35437)
0929 10:59:11.475279 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:11.475282 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:11.475501 (+ 219us) server_negotiation.cc:407] Connection header received
0929 10:59:11.475973 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:11.475974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:11.475987 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:11.476037 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:11.476965 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:11.477481 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:11.478695 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:11.479007 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:11.479077 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:11.481449 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:11.481450 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:11.481451 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:11.481459 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:11.483166 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:11.483563 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:11.483565 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:11.483565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:11.483630 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:11.483867 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:11.483868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:11.483869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:11.484035 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:11.484114 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:11.484380 (+ 266us) server_negotiation.cc:299] Negotiation successful
0929 10:59:11.484480 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 10:59:11.485176 27350 heartbeater.cc:347] Connected to a master server at 127.26.120.254:35437
I20260929 10:59:11.485286 27350 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:11.485455 27350 heartbeater.cc:511] Master 127.26.120.254:35437 requested a full tablet report, sending...
I20260929 10:59:11.486132 27159 ts_manager.cc:194] Registered new tserver with Master: cad95d67753e4b0c889e9b8ef5e4098c (127.26.120.193:36163)
I20260929 10:59:11.486887 27159 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.193@KRBTEST.COM'} at 127.26.120.193:46543
WARNING: no policy specified for kudu/127.26.120.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab.
Entry for principal kudu/127.26.120.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab.
I20260929 10:59:11.504668 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.194:0
--local_ip_for_outbound_sockets=127.26.120.194
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:35437
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:37553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:11.589353 27355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:11.589598 27355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:11.589638 27355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:11.590852 27355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:11.590924 27355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:11.590971 27355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.194
I20260929 10:59:11.592499 27355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:37553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:35437
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.194
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:11.592770 27355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:11.593036 27355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:11.598310 27360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:11.598290 27361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:11.598412 27355 server_base.cc:1034] running on GCE node
W20260929 10:59:11.598402 27363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:11.598695 27355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:11.598944 27355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:11.600133 27355 hybrid_clock.cc:648] HybridClock initialized: now 1790679551600133 us; error 44 us; skew 500 ppm
Sep 29 10:59:11 dist-test-slave-2lzq krb5kdc[27114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679551, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:11.602712 27355 init.cc:377] Logged in from keytab as kudu/127.26.120.194@KRBTEST.COM (short username kudu)
I20260929 10:59:11.603164 27355 webserver.cc:460] Webserver started at http://127.26.120.194:36803/ using document root <none> and password file <none>
I20260929 10:59:11.603410 27355 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:11.603478 27355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:11.603628 27355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:11.604696 27355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-1/data/instance:
uuid: "f207b77868004ebaa26976b1ffe0413d"
format_stamp: "Formatted at 2026-09-29 10:59:11 on dist-test-slave-2lzq"
server_key: "39eeaf004bbbdfe67898fa1994a82bb2"
server_key_iv: "952642b95471fd381c15134dc3e70c87"
server_key_version: "encryptionkey@0"
I20260929 10:59:11.605050 27355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance:
uuid: "f207b77868004ebaa26976b1ffe0413d"
format_stamp: "Formatted at 2026-09-29 10:59:11 on dist-test-slave-2lzq"
server_key: "39eeaf004bbbdfe67898fa1994a82bb2"
server_key_iv: "952642b95471fd381c15134dc3e70c87"
server_key_version: "encryptionkey@0"
I20260929 10:59:11.606464 27355 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 10:59:11.607167 27370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:11.607342 27355 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:59:11.607439 27355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-1/wal
uuid: "f207b77868004ebaa26976b1ffe0413d"
format_stamp: "Formatted at 2026-09-29 10:59:11 on dist-test-slave-2lzq"
server_key: "39eeaf004bbbdfe67898fa1994a82bb2"
server_key_iv: "952642b95471fd381c15134dc3e70c87"
server_key_version: "encryptionkey@0"
I20260929 10:59:11.607515 27355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:11.619567 27355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:11.621346 27355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:11.621515 27355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:11.621742 27355 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:59:11.622040 27355 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:11.622143 27355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:11.622184 27355 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:11.622211 27355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:11.629055 27355 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.194:43903
I20260929 10:59:11.629099 27483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.194:43903 every 8 connection(s)
I20260929 10:59:11.629449 27355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
Sep 29 10:59:11 dist-test-slave-2lzq krb5kdc[27114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679551, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:11.638289 27198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:11.630255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:51733 (local address 127.26.120.254:35437)
0929 10:59:11.630299 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:11.630301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:11.630554 (+ 253us) server_negotiation.cc:407] Connection header received
0929 10:59:11.631005 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:11.631006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:11.631018 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:11.631072 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:11.631988 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:11.632480 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:11.633443 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:11.633636 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:11.633687 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:11.635726 (+ 2039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:11.635728 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:11.635728 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:11.635734 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:11.637125 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:11.637541 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:11.637542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:11.637542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:11.637590 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:11.637793 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:11.637794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:11.637794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:11.637890 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:11.637938 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:11.638166 (+ 228us) server_negotiation.cc:299] Negotiation successful
0929 10:59:11.638226 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 10:59:11.638551 27486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:11.630229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:35437 (local address 127.26.120.194:51733)
0929 10:59:11.630414 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:11.630449 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:11.630916 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:11.631126 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:11.631129 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:11.631703 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:11.631897 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:11.631902 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:11.632565 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:11.632566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:11.633365 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:11.633367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:11.633707 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:11.633708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:11.633804 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:11.634272 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:11.634281 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:11.635641 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:11.637202 (+ 1561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:11.637208 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:11.637209 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:11.637456 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:11.637648 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:11.637648 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:11.637649 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:11.637739 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:11.637944 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:11.637948 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:11.638069 (+ 121us) client_negotiation.cc:769] Sending connection context
0929 10:59:11.638150 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 10:59:11.638260 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":116,"threads_started":1}
I20260929 10:59:11.639097 27484 heartbeater.cc:347] Connected to a master server at 127.26.120.254:35437
I20260929 10:59:11.639185 27484 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:11.639374 27484 heartbeater.cc:511] Master 127.26.120.254:35437 requested a full tablet report, sending...
I20260929 10:59:11.639418 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 27355
I20260929 10:59:11.639495 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance
I20260929 10:59:11.639676 27107 external_mini_cluster.cc:1428] Setting key 13c4852a6191f5cc52b2d033be820198
I20260929 10:59:11.639807 27159 ts_manager.cc:194] Registered new tserver with Master: f207b77868004ebaa26976b1ffe0413d (127.26.120.194:43903)
I20260929 10:59:11.640269 27159 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.194@KRBTEST.COM'} at 127.26.120.194:51733
WARNING: no policy specified for kudu/127.26.120.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab.
Entry for principal kudu/127.26.120.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab.
I20260929 10:59:11.664664 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.195:0
--local_ip_for_outbound_sockets=127.26.120.195
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:35437
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:37553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:11.746230 27489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:11.746424 27489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:11.746447 27489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:11.747614 27489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:11.747658 27489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:11.747684 27489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.195
I20260929 10:59:11.749329 27489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:37553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:35437
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.195
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:11.749576 27489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:11.749822 27489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:11.754791 27494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:11.754818 27497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:11.754801 27495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:11.755148 27489 server_base.cc:1034] running on GCE node
I20260929 10:59:11.755347 27489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:11.755600 27489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:11.756745 27489 hybrid_clock.cc:648] HybridClock initialized: now 1790679551756718 us; error 42 us; skew 500 ppm
Sep 29 10:59:11 dist-test-slave-2lzq krb5kdc[27114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679551, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:11.763195 27489 init.cc:377] Logged in from keytab as kudu/127.26.120.195@KRBTEST.COM (short username kudu)
I20260929 10:59:11.763727 27489 webserver.cc:460] Webserver started at http://127.26.120.195:42193/ using document root <none> and password file <none>
I20260929 10:59:11.764014 27489 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:11.764068 27489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:11.764254 27489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:11.765365 27489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-2/data/instance:
uuid: "5cd383e0084c485e977ec46ec44716fa"
format_stamp: "Formatted at 2026-09-29 10:59:11 on dist-test-slave-2lzq"
server_key: "3a146d7537c017477a4491fb74f1d2a2"
server_key_iv: "a02b24818fae0399ce585dfa9e77dca7"
server_key_version: "encryptionkey@0"
I20260929 10:59:11.765738 27489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance:
uuid: "5cd383e0084c485e977ec46ec44716fa"
format_stamp: "Formatted at 2026-09-29 10:59:11 on dist-test-slave-2lzq"
server_key: "3a146d7537c017477a4491fb74f1d2a2"
server_key_iv: "a02b24818fae0399ce585dfa9e77dca7"
server_key_version: "encryptionkey@0"
I20260929 10:59:11.767234 27489 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:59:11.768009 27504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:11.768195 27489 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:59:11.768301 27489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-2/wal
uuid: "5cd383e0084c485e977ec46ec44716fa"
format_stamp: "Formatted at 2026-09-29 10:59:11 on dist-test-slave-2lzq"
server_key: "3a146d7537c017477a4491fb74f1d2a2"
server_key_iv: "a02b24818fae0399ce585dfa9e77dca7"
server_key_version: "encryptionkey@0"
I20260929 10:59:11.768394 27489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:11.776187 27489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:11.777998 27489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:11.778192 27489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:11.778417 27489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:59:11.778729 27489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:11.778787 27489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:11.778827 27489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:11.778885 27489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:11.785602 27489 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.195:39503
I20260929 10:59:11.785638 27617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.195:39503 every 8 connection(s)
I20260929 10:59:11.786121 27489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
I20260929 10:59:11.789225 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 27489
I20260929 10:59:11.789320 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance
I20260929 10:59:11.789502 27107 external_mini_cluster.cc:1428] Setting key 103e475f1dea3d6d506ebbd15edbf888
Sep 29 10:59:11 dist-test-slave-2lzq krb5kdc[27114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679551, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:11.795920 27198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:11.786966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:53821 (local address 127.26.120.254:35437)
0929 10:59:11.787024 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:11.787026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:11.787411 (+ 385us) server_negotiation.cc:407] Connection header received
0929 10:59:11.787885 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:11.787886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:11.787897 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:11.787954 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:11.788865 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:11.789566 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:11.790926 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:11.791156 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:11.791251 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:11.793386 (+ 2135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:11.793388 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:11.793388 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:11.793395 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:11.794740 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:11.795116 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:11.795117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:11.795117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:11.795175 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:11.795400 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:11.795401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:11.795401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:11.795507 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:11.795561 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:11.795788 (+ 227us) server_negotiation.cc:299] Negotiation successful
0929 10:59:11.795860 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 10:59:11.796073 27620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:11.786974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:35437 (local address 127.26.120.195:53821)
0929 10:59:11.787172 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:11.787280 (+ 108us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:11.787799 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:11.788005 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:11.788008 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:11.788571 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:11.788764 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:11.788769 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:11.789659 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:11.789660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:11.790812 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:11.790814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:11.791260 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:11.791261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:11.791423 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:11.791891 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:11.791899 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:11.793303 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:11.794812 (+ 1509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:11.794817 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:11.794819 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:11.795034 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:11.795244 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:11.795244 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:11.795244 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:11.795339 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:11.795565 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:11.795569 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:11.795678 (+ 109us) client_negotiation.cc:769] Sending connection context
0929 10:59:11.795759 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 10:59:11.795874 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":122,"threads_started":1}
I20260929 10:59:11.796536 27618 heartbeater.cc:347] Connected to a master server at 127.26.120.254:35437
I20260929 10:59:11.796639 27618 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:11.796828 27618 heartbeater.cc:511] Master 127.26.120.254:35437 requested a full tablet report, sending...
I20260929 10:59:11.797251 27158 ts_manager.cc:194] Registered new tserver with Master: 5cd383e0084c485e977ec46ec44716fa (127.26.120.195:39503)
I20260929 10:59:11.797734 27158 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.195@KRBTEST.COM'} at 127.26.120.195:53821
I20260929 10:59:11.801492 27107 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 10:59:11 dist-test-slave-2lzq krb5kdc[27114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679551, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 10:59:11 dist-test-slave-2lzq krb5kdc[27114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679551, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.120.193@KRBTEST.COM
I20260929 10:59:11.825248 27625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:11.817461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32840 (local address 127.26.120.193:36163)
0929 10:59:11.817641 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:11.817646 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:11.817658 (+ 12us) server_negotiation.cc:407] Connection header received
0929 10:59:11.817684 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:11.817686 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:11.817700 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:11.817768 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:11.818417 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:11.819157 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:11.819842 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:11.820162 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:11.820218 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:11.822579 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:11.822587 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:11.822587 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:11.822594 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:11.824046 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:11.824376 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:11.824379 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:11.824379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:11.824444 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:11.824647 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:11.824648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:11.824648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:11.824839 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:11.824971 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:11.825016 (+ 45us) server_negotiation.cc:299] Negotiation successful
0929 10:59:11.825088 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":96,"threads_started":1}
W20260929 10:59:11.825739 27264 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32840
Sep 29 10:59:11 dist-test-slave-2lzq krb5kdc[27114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 10:59:11 dist-test-slave-2lzq krb5kdc[27114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.193@KRBTEST.COM
I20260929 10:59:11.848505 27625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:11.841178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32850 (local address 127.26.120.193:36163)
0929 10:59:11.841236 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:11.841238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:11.841369 (+ 131us) server_negotiation.cc:407] Connection header received
0929 10:59:11.841435 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:11.841436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:11.841455 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:11.841515 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:11.842094 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:11.842598 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:11.843344 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:11.843551 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:11.843628 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:11.845968 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:11.845969 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:11.845970 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:11.845976 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:11.847433 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:11.847760 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:11.847761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:11.847761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:11.847826 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:11.847989 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:11.847989 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:11.847990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:11.848081 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:11.848176 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:11.848256 (+ 80us) server_negotiation.cc:299] Negotiation successful
0929 10:59:11.848352 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 10:59:11.849056 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 27221
I20260929 10:59:11.853657 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 27355
I20260929 10:59:11.857950 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 27489
I20260929 10:59:11.861888 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 27127
2026-09-29T10:59:11Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1881 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:59:11 dist-test-slave-2lzq krb5kdc[27632](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:59:11 dist-test-slave-2lzq krb5kdc[27632](info): set up 2 sockets
Sep 29 10:59:11 dist-test-slave-2lzq krb5kdc[27632](info): commencing operation
krb5kdc: starting...
W20260929 10:59:12.889645 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 10:59:12 dist-test-slave-2lzq krb5kdc[27632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679552, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:59:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:59:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.120.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
I20260929 10:59:13.008569 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:34281
--webserver_interface=127.26.120.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:43931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:34281
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:13.087841 27645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:13.088054 27645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:13.088076 27645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:13.089159 27645 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:59:13.089192 27645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:13.089206 27645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:13.089219 27645 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:59:13.089231 27645 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:59:13.090659 27645 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:43931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:34281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:34281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:13.090862 27645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:13.091032 27645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:13.095845 27653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:13.095876 27651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:13.095821 27650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:13.096098 27645 server_base.cc:1034] running on GCE node
I20260929 10:59:13.096433 27645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:13.096787 27645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:13.097966 27645 hybrid_clock.cc:648] HybridClock initialized: now 1790679553097934 us; error 48 us; skew 500 ppm
Sep 29 10:59:13 dist-test-slave-2lzq krb5kdc[27632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679553, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:13.101029 27645 init.cc:377] Logged in from keytab as kudu/127.26.120.254@KRBTEST.COM (short username kudu)
I20260929 10:59:13.101430 27645 webserver.cc:460] Webserver started at http://127.26.120.254:37313/ using document root <none> and password file <none>
I20260929 10:59:13.101624 27645 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:13.101666 27645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:13.101753 27645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:13.102821 27645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "6f368ac125dd4610a69a1084a8bfa624"
format_stamp: "Formatted at 2026-09-29 10:59:13 on dist-test-slave-2lzq"
server_key: "a134d22f134e394c1dfec7fd7aa45653"
server_key_iv: "05d034c2d043ef3c11d8b8908b5a8df5"
server_key_version: "encryptionkey@0"
I20260929 10:59:13.103197 27645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "6f368ac125dd4610a69a1084a8bfa624"
format_stamp: "Formatted at 2026-09-29 10:59:13 on dist-test-slave-2lzq"
server_key: "a134d22f134e394c1dfec7fd7aa45653"
server_key_iv: "05d034c2d043ef3c11d8b8908b5a8df5"
server_key_version: "encryptionkey@0"
I20260929 10:59:13.104489 27645 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 10:59:13.105183 27660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:13.105383 27645 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:59:13.105455 27645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "6f368ac125dd4610a69a1084a8bfa624"
format_stamp: "Formatted at 2026-09-29 10:59:13 on dist-test-slave-2lzq"
server_key: "a134d22f134e394c1dfec7fd7aa45653"
server_key_iv: "05d034c2d043ef3c11d8b8908b5a8df5"
server_key_version: "encryptionkey@0"
I20260929 10:59:13.105548 27645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:13.113066 27645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:13.115240 27645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:13.115410 27645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:13.119493 27645 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:34281
I20260929 10:59:13.119530 27712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:34281 every 8 connection(s)
I20260929 10:59:13.119879 27645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 10:59:13.120756 27713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:13.123103 27713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f368ac125dd4610a69a1084a8bfa624: Bootstrap starting.
I20260929 10:59:13.123417 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 27645
I20260929 10:59:13.123486 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 10:59:13.123656 27107 external_mini_cluster.cc:1428] Setting key 8b1ef8053964136637d4edd7508e7c79
I20260929 10:59:13.123791 27713 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f368ac125dd4610a69a1084a8bfa624: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:13.124081 27713 log.cc:824] T 00000000000000000000000000000000 P 6f368ac125dd4610a69a1084a8bfa624: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:13.124871 27713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f368ac125dd4610a69a1084a8bfa624: No bootstrap required, opened a new log
I20260929 10:59:13.126801 27713 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f368ac125dd4610a69a1084a8bfa624 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f368ac125dd4610a69a1084a8bfa624" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34281 } }
I20260929 10:59:13.126961 27713 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f368ac125dd4610a69a1084a8bfa624 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:13.127013 27713 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f368ac125dd4610a69a1084a8bfa624 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f368ac125dd4610a69a1084a8bfa624, State: Initialized, Role: FOLLOWER
I20260929 10:59:13.127163 27713 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f368ac125dd4610a69a1084a8bfa624 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f368ac125dd4610a69a1084a8bfa624" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34281 } }
I20260929 10:59:13.127245 27713 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f368ac125dd4610a69a1084a8bfa624 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:59:13.127288 27713 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f368ac125dd4610a69a1084a8bfa624 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:59:13.127344 27713 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f368ac125dd4610a69a1084a8bfa624 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:13.127888 27713 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f368ac125dd4610a69a1084a8bfa624 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f368ac125dd4610a69a1084a8bfa624" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34281 } }
I20260929 10:59:13.128010 27713 leader_election.cc:302] T 00000000000000000000000000000000 P 6f368ac125dd4610a69a1084a8bfa624 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f368ac125dd4610a69a1084a8bfa624; no voters:
I20260929 10:59:13.128217 27713 leader_election.cc:288] T 00000000000000000000000000000000 P 6f368ac125dd4610a69a1084a8bfa624 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:59:13.128245 27718 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f368ac125dd4610a69a1084a8bfa624 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:59:13.128355 27718 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f368ac125dd4610a69a1084a8bfa624 [term 1 LEADER]: Becoming Leader. State: Replica: 6f368ac125dd4610a69a1084a8bfa624, State: Running, Role: LEADER
I20260929 10:59:13.128436 27718 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f368ac125dd4610a69a1084a8bfa624 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f368ac125dd4610a69a1084a8bfa624" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34281 } }
I20260929 10:59:13.128484 27713 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f368ac125dd4610a69a1084a8bfa624 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:59:13.129135 27718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f368ac125dd4610a69a1084a8bfa624 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f368ac125dd4610a69a1084a8bfa624" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f368ac125dd4610a69a1084a8bfa624" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34281 } } }
I20260929 10:59:13.129232 27718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f368ac125dd4610a69a1084a8bfa624 [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:13.129361 27718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f368ac125dd4610a69a1084a8bfa624 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f368ac125dd4610a69a1084a8bfa624. Latest consensus state: current_term: 1 leader_uuid: "6f368ac125dd4610a69a1084a8bfa624" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f368ac125dd4610a69a1084a8bfa624" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34281 } } }
I20260929 10:59:13.129416 27718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f368ac125dd4610a69a1084a8bfa624 [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:13.129660 27727 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:59:13.130261 27727 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 29 10:59:13 dist-test-slave-2lzq krb5kdc[27632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679552, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:13.131899 27727 catalog_manager.cc:1347] Generated new cluster ID: 9d15d3443d5d4bb581d08da9f8b04cbc
I20260929 10:59:13.131948 27727 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:59:13.137519 27717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:13.124202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53876 (local address 127.26.120.254:34281)
0929 10:59:13.124419 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:13.124428 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:13.124492 (+ 64us) server_negotiation.cc:407] Connection header received
0929 10:59:13.125093 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:13.125101 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:13.125151 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:13.125296 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:13.125912 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.126840 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.127798 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.128329 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.128438 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:13.131464 (+ 3026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:13.131467 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:13.131468 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:13.131477 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:13.135657 (+ 4180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:13.136048 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:13.136051 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:13.136052 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:13.136145 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:13.136305 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:13.136306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:13.136306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:13.136578 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:13.136715 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:13.137166 (+ 451us) server_negotiation.cc:299] Negotiation successful
0929 10:59:13.137306 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":96,"threads_started":1}
I20260929 10:59:13.142397 27727 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:59:13.143098 27727 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:59:13.149828 27727 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f368ac125dd4610a69a1084a8bfa624: Generated new TSK 0
I20260929 10:59:13.150007 27727 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.120.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab.
Entry for principal kudu/127.26.120.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab.
I20260929 10:59:13.182189 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.193:0
--local_ip_for_outbound_sockets=127.26.120.193
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:34281
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:43931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:13.262982 27739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:13.263175 27739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:13.263198 27739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:13.264405 27739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:13.264448 27739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:13.264475 27739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.193
I20260929 10:59:13.266019 27739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:43931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:34281
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.193
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:13.266310 27739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:13.266604 27739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:13.271477 27745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:13.271473 27747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:13.271509 27739 server_base.cc:1034] running on GCE node
W20260929 10:59:13.271478 27744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:13.271898 27739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:13.272138 27739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:13.273278 27739 hybrid_clock.cc:648] HybridClock initialized: now 1790679553273266 us; error 29 us; skew 500 ppm
Sep 29 10:59:13 dist-test-slave-2lzq krb5kdc[27632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679553, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:13.275911 27739 init.cc:377] Logged in from keytab as kudu/127.26.120.193@KRBTEST.COM (short username kudu)
I20260929 10:59:13.276322 27739 webserver.cc:460] Webserver started at http://127.26.120.193:35493/ using document root <none> and password file <none>
I20260929 10:59:13.276577 27739 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:13.276647 27739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:13.276808 27739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:13.277905 27739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-0/data/instance:
uuid: "73c0d747796c4975b83b7648a8ae6abd"
format_stamp: "Formatted at 2026-09-29 10:59:13 on dist-test-slave-2lzq"
server_key: "99a2ab038141d163c38889bc0b78f47e"
server_key_iv: "643924b749e2387391e6c5d67ddfde26"
server_key_version: "encryptionkey@0"
I20260929 10:59:13.278292 27739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance:
uuid: "73c0d747796c4975b83b7648a8ae6abd"
format_stamp: "Formatted at 2026-09-29 10:59:13 on dist-test-slave-2lzq"
server_key: "99a2ab038141d163c38889bc0b78f47e"
server_key_iv: "643924b749e2387391e6c5d67ddfde26"
server_key_version: "encryptionkey@0"
I20260929 10:59:13.279716 27739 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:59:13.280463 27754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:13.280692 27739 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:59:13.280833 27739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-0/wal
uuid: "73c0d747796c4975b83b7648a8ae6abd"
format_stamp: "Formatted at 2026-09-29 10:59:13 on dist-test-slave-2lzq"
server_key: "99a2ab038141d163c38889bc0b78f47e"
server_key_iv: "643924b749e2387391e6c5d67ddfde26"
server_key_version: "encryptionkey@0"
I20260929 10:59:13.280920 27739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:13.291934 27739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:13.293850 27739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:13.294014 27739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:13.294246 27739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:59:13.294564 27739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:13.294620 27739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:13.294658 27739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:13.294687 27739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:13.300952 27739 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.193:35487
I20260929 10:59:13.300997 27867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.193:35487 every 8 connection(s)
I20260929 10:59:13.301293 27739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
I20260929 10:59:13.307152 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 27739
I20260929 10:59:13.307246 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance
Sep 29 10:59:13 dist-test-slave-2lzq krb5kdc[27632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679553, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:13.307425 27107 external_mini_cluster.cc:1428] Setting key b3888129ab6bfb49e9a2a3962152de54
I20260929 10:59:13.310882 27717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:13.301991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:39411 (local address 127.26.120.254:34281)
0929 10:59:13.302103 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:13.302106 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:13.302246 (+ 140us) server_negotiation.cc:407] Connection header received
0929 10:59:13.303103 (+ 857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:13.303104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:13.303113 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:13.303160 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:13.303722 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.304202 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.305235 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.305443 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.305499 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:13.307860 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:13.307861 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:13.307861 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:13.307868 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:13.309440 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:13.309844 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:13.309845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:13.309846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:13.309908 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:13.310228 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:13.310230 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:13.310230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:13.310352 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:13.310433 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:13.310709 (+ 276us) server_negotiation.cc:299] Negotiation successful
0929 10:59:13.310793 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 10:59:13.311064 27870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:13.301952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34281 (local address 127.26.120.193:39411)
0929 10:59:13.302124 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:13.302167 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:13.303015 (+ 848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:13.303220 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:13.303224 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:13.303472 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:13.303654 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:13.303657 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.304296 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.304333 (+ 37us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:13.305158 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:13.305160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.305575 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.305576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:13.305692 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:13.306212 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:13.306220 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:13.307766 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:13.309534 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:13.309541 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:13.309543 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:13.309763 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:13.309982 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:13.309983 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:13.309983 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:13.310154 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:13.310422 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:13.310429 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:13.310575 (+ 146us) client_negotiation.cc:769] Sending connection context
0929 10:59:13.310676 (+ 101us) client_negotiation.cc:240] Negotiation successful
0929 10:59:13.310829 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":108,"threads_started":1}
I20260929 10:59:13.311687 27868 heartbeater.cc:347] Connected to a master server at 127.26.120.254:34281
I20260929 10:59:13.311769 27868 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:13.311959 27868 heartbeater.cc:511] Master 127.26.120.254:34281 requested a full tablet report, sending...
I20260929 10:59:13.312486 27677 ts_manager.cc:194] Registered new tserver with Master: 73c0d747796c4975b83b7648a8ae6abd (127.26.120.193:35487)
I20260929 10:59:13.313108 27677 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.193@KRBTEST.COM'} at 127.26.120.193:39411
WARNING: no policy specified for kudu/127.26.120.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab.
Entry for principal kudu/127.26.120.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab.
I20260929 10:59:13.332154 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.194:0
--local_ip_for_outbound_sockets=127.26.120.194
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:34281
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:43931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:13.411866 27873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:13.412061 27873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:13.412086 27873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:13.413266 27873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:13.413312 27873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:13.413369 27873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.194
I20260929 10:59:13.414969 27873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:43931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:34281
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.194
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:13.415216 27873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:13.415462 27873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:13.420449 27881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:13.420559 27878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:13.420680 27873 server_base.cc:1034] running on GCE node
W20260929 10:59:13.420567 27879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:13.420964 27873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:13.421222 27873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:13.422406 27873 hybrid_clock.cc:648] HybridClock initialized: now 1790679553422420 us; error 71 us; skew 500 ppm
Sep 29 10:59:13 dist-test-slave-2lzq krb5kdc[27632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679553, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:13.424964 27873 init.cc:377] Logged in from keytab as kudu/127.26.120.194@KRBTEST.COM (short username kudu)
I20260929 10:59:13.425374 27873 webserver.cc:460] Webserver started at http://127.26.120.194:34999/ using document root <none> and password file <none>
I20260929 10:59:13.425590 27873 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:13.425657 27873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:13.425810 27873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:13.426903 27873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-1/data/instance:
uuid: "0ff57111a35f4adaa4a5fb98b4d7dece"
format_stamp: "Formatted at 2026-09-29 10:59:13 on dist-test-slave-2lzq"
server_key: "e822e1970fb961ae34abfb58b31926ad"
server_key_iv: "f707543c54e837e3fdb6c50b6bcf02b1"
server_key_version: "encryptionkey@0"
I20260929 10:59:13.427268 27873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance:
uuid: "0ff57111a35f4adaa4a5fb98b4d7dece"
format_stamp: "Formatted at 2026-09-29 10:59:13 on dist-test-slave-2lzq"
server_key: "e822e1970fb961ae34abfb58b31926ad"
server_key_iv: "f707543c54e837e3fdb6c50b6bcf02b1"
server_key_version: "encryptionkey@0"
I20260929 10:59:13.428668 27873 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 10:59:13.429386 27888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:13.429602 27873 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:59:13.429682 27873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-1/wal
uuid: "0ff57111a35f4adaa4a5fb98b4d7dece"
format_stamp: "Formatted at 2026-09-29 10:59:13 on dist-test-slave-2lzq"
server_key: "e822e1970fb961ae34abfb58b31926ad"
server_key_iv: "f707543c54e837e3fdb6c50b6bcf02b1"
server_key_version: "encryptionkey@0"
I20260929 10:59:13.429756 27873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:13.459137 27873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:13.461041 27873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:13.461207 27873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:13.461483 27873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:59:13.461797 27873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:13.461859 27873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:13.461900 27873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:13.461930 27873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:13.467685 27873 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.194:41849
I20260929 10:59:13.467728 28001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.194:41849 every 8 connection(s)
I20260929 10:59:13.468140 27873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
Sep 29 10:59:13 dist-test-slave-2lzq krb5kdc[27632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679553, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:13.476899 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 27873
I20260929 10:59:13.477018 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance
I20260929 10:59:13.477205 27107 external_mini_cluster.cc:1428] Setting key c208cbbd25934b841e81d17299330c87
I20260929 10:59:13.477743 27717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:13.468897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:40085 (local address 127.26.120.254:34281)
0929 10:59:13.468986 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:13.468989 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:13.469168 (+ 179us) server_negotiation.cc:407] Connection header received
0929 10:59:13.470150 (+ 982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:13.470151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:13.470163 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:13.470216 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:13.470837 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.471344 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.472370 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.472580 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.472665 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:13.474747 (+ 2082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:13.474749 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:13.474749 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:13.474757 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:13.476035 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:13.476475 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:13.476477 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:13.476477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:13.476560 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:13.476926 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:13.476927 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:13.476927 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:13.477092 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:13.477208 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:13.477600 (+ 392us) server_negotiation.cc:299] Negotiation successful
0929 10:59:13.477679 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 10:59:13.477968 28004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:13.468867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34281 (local address 127.26.120.194:40085)
0929 10:59:13.469042 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:13.469076 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:13.469994 (+ 918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:13.470290 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:13.470294 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:13.470549 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:13.470730 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:13.470734 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.471437 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.471438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:13.472250 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:13.472252 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.472673 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.472675 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:13.472794 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:13.473235 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:13.473243 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:13.474650 (+ 1407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:13.476147 (+ 1497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:13.476154 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:13.476156 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:13.476380 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:13.476650 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:13.476651 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:13.476651 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:13.476842 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:13.477208 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:13.477216 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:13.477430 (+ 214us) client_negotiation.cc:769] Sending connection context
0929 10:59:13.477568 (+ 138us) client_negotiation.cc:240] Negotiation successful
0929 10:59:13.477747 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":81,"threads_started":1}
I20260929 10:59:13.478436 28002 heartbeater.cc:347] Connected to a master server at 127.26.120.254:34281
I20260929 10:59:13.478518 28002 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:13.478701 28002 heartbeater.cc:511] Master 127.26.120.254:34281 requested a full tablet report, sending...
I20260929 10:59:13.479048 27677 ts_manager.cc:194] Registered new tserver with Master: 0ff57111a35f4adaa4a5fb98b4d7dece (127.26.120.194:41849)
I20260929 10:59:13.479388 27677 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.194@KRBTEST.COM'} at 127.26.120.194:40085
WARNING: no policy specified for kudu/127.26.120.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab.
Entry for principal kudu/127.26.120.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab.
I20260929 10:59:13.500869 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.195:0
--local_ip_for_outbound_sockets=127.26.120.195
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:34281
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:43931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:13.582815 28007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:13.583021 28007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:13.583045 28007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:13.584309 28007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:13.584352 28007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:13.584399 28007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.195
I20260929 10:59:13.585964 28007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:43931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:34281
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.195
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:13.586253 28007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:13.586503 28007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:13.591339 28015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:13.591339 28012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:13.591341 28013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:13.591384 28007 server_base.cc:1034] running on GCE node
I20260929 10:59:13.591730 28007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:13.591982 28007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:13.593127 28007 hybrid_clock.cc:648] HybridClock initialized: now 1790679553593120 us; error 31 us; skew 500 ppm
Sep 29 10:59:13 dist-test-slave-2lzq krb5kdc[27632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679553, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:13.595732 28007 init.cc:377] Logged in from keytab as kudu/127.26.120.195@KRBTEST.COM (short username kudu)
I20260929 10:59:13.596117 28007 webserver.cc:460] Webserver started at http://127.26.120.195:45575/ using document root <none> and password file <none>
I20260929 10:59:13.596303 28007 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:13.596341 28007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:13.596427 28007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:13.597400 28007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-2/data/instance:
uuid: "23f0f05beebe49ed80beab70ee08c4ee"
format_stamp: "Formatted at 2026-09-29 10:59:13 on dist-test-slave-2lzq"
server_key: "a9203b4172bb7a0359128daac2b7018c"
server_key_iv: "3372fcd4ebf77f7e6d9c417be33ceb58"
server_key_version: "encryptionkey@0"
I20260929 10:59:13.597697 28007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance:
uuid: "23f0f05beebe49ed80beab70ee08c4ee"
format_stamp: "Formatted at 2026-09-29 10:59:13 on dist-test-slave-2lzq"
server_key: "a9203b4172bb7a0359128daac2b7018c"
server_key_iv: "3372fcd4ebf77f7e6d9c417be33ceb58"
server_key_version: "encryptionkey@0"
I20260929 10:59:13.599016 28007 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:59:13.599670 28022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:13.599858 28007 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:59:13.599941 28007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-2/wal
uuid: "23f0f05beebe49ed80beab70ee08c4ee"
format_stamp: "Formatted at 2026-09-29 10:59:13 on dist-test-slave-2lzq"
server_key: "a9203b4172bb7a0359128daac2b7018c"
server_key_iv: "3372fcd4ebf77f7e6d9c417be33ceb58"
server_key_version: "encryptionkey@0"
I20260929 10:59:13.599996 28007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:13.613793 28007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:13.615725 28007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:13.615830 28007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:13.615993 28007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:59:13.616231 28007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:13.616259 28007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:13.616279 28007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:13.616292 28007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:13.621833 28007 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.195:35459
I20260929 10:59:13.621909 28135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.195:35459 every 8 connection(s)
I20260929 10:59:13.622262 28007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
I20260929 10:59:13.625365 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 28007
I20260929 10:59:13.625459 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance
I20260929 10:59:13.625638 27107 external_mini_cluster.cc:1428] Setting key 830a116b589150297338a780e89d2ba6
Sep 29 10:59:13 dist-test-slave-2lzq krb5kdc[27632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679553, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:13.631947 27717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:13.622974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:51761 (local address 127.26.120.254:34281)
0929 10:59:13.623040 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:13.623042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:13.623263 (+ 221us) server_negotiation.cc:407] Connection header received
0929 10:59:13.624220 (+ 957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:13.624221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:13.624235 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:13.624286 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:13.624914 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.625616 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.626929 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.627120 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.627175 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:13.629302 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:13.629303 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:13.629304 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:13.629310 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:13.630847 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:13.631200 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:13.631201 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:13.631201 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:13.631249 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:13.631462 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:13.631463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:13.631463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:13.631560 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:13.631610 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:13.631810 (+ 200us) server_negotiation.cc:299] Negotiation successful
0929 10:59:13.631884 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 10:59:13.632179 28138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:13.622967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34281 (local address 127.26.120.195:51761)
0929 10:59:13.623149 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:13.623183 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:13.624102 (+ 919us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:13.624361 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:13.624366 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:13.624622 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:13.624807 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:13.624811 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.625724 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.625725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:13.626857 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:13.626859 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.627191 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.627191 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:13.627294 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:13.627714 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:13.627721 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:13.629205 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:13.630915 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:13.630921 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:13.630923 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:13.631136 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:13.631312 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:13.631313 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:13.631313 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:13.631404 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:13.631612 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:13.631616 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:13.631721 (+ 105us) client_negotiation.cc:769] Sending connection context
0929 10:59:13.631801 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 10:59:13.631915 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":91,"threads_started":1}
I20260929 10:59:13.632681 28136 heartbeater.cc:347] Connected to a master server at 127.26.120.254:34281
I20260929 10:59:13.632764 28136 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:13.632936 28136 heartbeater.cc:511] Master 127.26.120.254:34281 requested a full tablet report, sending...
I20260929 10:59:13.633391 27677 ts_manager.cc:194] Registered new tserver with Master: 23f0f05beebe49ed80beab70ee08c4ee (127.26.120.195:35459)
I20260929 10:59:13.633787 27677 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.195@KRBTEST.COM'} at 127.26.120.195:51761
I20260929 10:59:13.637192 27107 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:59:13.644197 27717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:13.638807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53886 (local address 127.26.120.254:34281)
0929 10:59:13.638862 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:13.638864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:13.638945 (+ 81us) server_negotiation.cc:407] Connection header received
0929 10:59:13.638990 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:13.638991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:13.638999 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:13.639059 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:13.639565 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.640010 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.640807 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.640977 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.641036 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:13.642126 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:13.642127 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:13.642128 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:13.642134 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:13.643258 (+ 1124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:13.643604 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:13.643605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:13.643605 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:13.643651 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:13.643809 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:13.643809 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:13.643810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:13.643929 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:13.644001 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:13.644082 (+ 81us) server_negotiation.cc:299] Negotiation successful
0929 10:59:13.644138 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 10:59:13.646235 27677 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53886:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:59:13.647955 27677 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:59:13.654537 28145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:13.650330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:41849 (local address 127.0.0.1:42884)
0929 10:59:13.650717 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:13.650729 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:13.650796 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:13.651062 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:13.651063 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:13.651064 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:13.651126 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:13.651127 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.652441 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.652442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:13.653263 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:13.653265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.654213 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.654214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:13.654296 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:13.654321 (+ 25us) client_negotiation.cc:769] Sending connection context
0929 10:59:13.654391 (+ 70us) client_negotiation.cc:240] Negotiation successful
0929 10:59:13.654445 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"mutex_wait_us":2,"thread_start_us":67,"threads_started":1}
I20260929 10:59:13.654549 28148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:13.650550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42884 (local address 127.26.120.194:41849)
0929 10:59:13.650714 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:13.650717 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:13.650806 (+ 89us) server_negotiation.cc:407] Connection header received
0929 10:59:13.650882 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:13.650885 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:13.650900 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:13.650971 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:13.651210 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.652336 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.653350 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.654112 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.654187 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:13.654243 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:13.654421 (+ 178us) server_negotiation.cc:299] Negotiation successful
0929 10:59:13.654471 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":82,"threads_started":1}
I20260929 10:59:13.654717 28147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:13.650541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35952 (local address 127.26.120.195:35459)
0929 10:59:13.650794 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:13.650798 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:13.650807 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:59:13.650848 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:13.650850 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:13.650867 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:13.650947 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:13.651275 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.652306 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.653618 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.654228 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.654295 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:13.654348 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:13.654580 (+ 232us) server_negotiation.cc:299] Negotiation successful
0929 10:59:13.654630 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":95,"threads_started":1}
I20260929 10:59:13.654881 28149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:13.650730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34296 (local address 127.26.120.193:35487)
0929 10:59:13.650928 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:13.650933 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:13.650944 (+ 11us) server_negotiation.cc:407] Connection header received
0929 10:59:13.651010 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:13.651012 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:13.651026 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:13.651122 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:13.651233 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.652034 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.653471 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.654194 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.654257 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:13.654304 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:13.654578 (+ 274us) server_negotiation.cc:299] Negotiation successful
0929 10:59:13.654754 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":87,"threads_started":1}
I20260929 10:59:13.655012 28146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:13.650330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:35487 (local address 127.0.0.1:34296)
0929 10:59:13.650890 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:13.650899 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:13.650938 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:13.651120 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:13.651121 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:13.651123 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:13.651177 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:13.651179 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.652138 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.652139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:13.653372 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:13.653375 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.654335 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.654336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:13.654422 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:13.654427 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 10:59:13.654479 (+ 52us) client_negotiation.cc:240] Negotiation successful
0929 10:59:13.654518 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"mutex_wait_us":3,"thread_start_us":67,"threads_started":1}
I20260929 10:59:13.655110 28144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:13.650313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:35459 (local address 127.0.0.1:35952)
0929 10:59:13.650400 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:13.650438 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:13.650502 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:13.651047 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:13.651051 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:13.651054 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:13.651134 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:13.651135 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.652437 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.652438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:13.653543 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:13.653544 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.654381 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.654382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:13.654446 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:13.654449 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:59:13.654506 (+ 57us) client_negotiation.cc:240] Negotiation successful
0929 10:59:13.654540 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63,"mutex_wait_us":37,"thread_start_us":45,"threads_started":1}
I20260929 10:59:13.655405 27936 tablet_service.cc:1467] Processing CreateTablet for tablet 9929a9b125de4b04948771f268534d20 (DEFAULT_TABLE table=test-table [id=d379b5a0efb3437286530fbb47d2d1a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:13.655674 27802 tablet_service.cc:1467] Processing CreateTablet for tablet 9929a9b125de4b04948771f268534d20 (DEFAULT_TABLE table=test-table [id=d379b5a0efb3437286530fbb47d2d1a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:13.655779 27936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9929a9b125de4b04948771f268534d20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:13.655814 28070 tablet_service.cc:1467] Processing CreateTablet for tablet 9929a9b125de4b04948771f268534d20 (DEFAULT_TABLE table=test-table [id=d379b5a0efb3437286530fbb47d2d1a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:13.655938 27802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9929a9b125de4b04948771f268534d20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:13.656086 28070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9929a9b125de4b04948771f268534d20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:13.659178 28151 tablet_bootstrap.cc:492] T 9929a9b125de4b04948771f268534d20 P 73c0d747796c4975b83b7648a8ae6abd: Bootstrap starting.
I20260929 10:59:13.659839 28151 tablet_bootstrap.cc:654] T 9929a9b125de4b04948771f268534d20 P 73c0d747796c4975b83b7648a8ae6abd: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:13.660445 28151 log.cc:824] T 9929a9b125de4b04948771f268534d20 P 73c0d747796c4975b83b7648a8ae6abd: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:13.660769 28150 tablet_bootstrap.cc:492] T 9929a9b125de4b04948771f268534d20 P 0ff57111a35f4adaa4a5fb98b4d7dece: Bootstrap starting.
I20260929 10:59:13.661203 28151 tablet_bootstrap.cc:492] T 9929a9b125de4b04948771f268534d20 P 73c0d747796c4975b83b7648a8ae6abd: No bootstrap required, opened a new log
I20260929 10:59:13.661259 28151 ts_tablet_manager.cc:1397] T 9929a9b125de4b04948771f268534d20 P 73c0d747796c4975b83b7648a8ae6abd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:59:13.661337 28150 tablet_bootstrap.cc:654] T 9929a9b125de4b04948771f268534d20 P 0ff57111a35f4adaa4a5fb98b4d7dece: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:13.661970 28152 tablet_bootstrap.cc:492] T 9929a9b125de4b04948771f268534d20 P 23f0f05beebe49ed80beab70ee08c4ee: Bootstrap starting.
I20260929 10:59:13.662554 28152 tablet_bootstrap.cc:654] T 9929a9b125de4b04948771f268534d20 P 23f0f05beebe49ed80beab70ee08c4ee: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:13.663012 28152 log.cc:824] T 9929a9b125de4b04948771f268534d20 P 23f0f05beebe49ed80beab70ee08c4ee: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:13.662935 28151 raft_consensus.cc:348] T 9929a9b125de4b04948771f268534d20 P 73c0d747796c4975b83b7648a8ae6abd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23f0f05beebe49ed80beab70ee08c4ee" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35459 } } peers { permanent_uuid: "73c0d747796c4975b83b7648a8ae6abd" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35487 } } peers { permanent_uuid: "0ff57111a35f4adaa4a5fb98b4d7dece" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41849 } }
I20260929 10:59:13.663094 28151 raft_consensus.cc:374] T 9929a9b125de4b04948771f268534d20 P 73c0d747796c4975b83b7648a8ae6abd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:13.663141 28151 raft_consensus.cc:729] T 9929a9b125de4b04948771f268534d20 P 73c0d747796c4975b83b7648a8ae6abd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73c0d747796c4975b83b7648a8ae6abd, State: Initialized, Role: FOLLOWER
I20260929 10:59:13.663261 28151 consensus_queue.cc:260] T 9929a9b125de4b04948771f268534d20 P 73c0d747796c4975b83b7648a8ae6abd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23f0f05beebe49ed80beab70ee08c4ee" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35459 } } peers { permanent_uuid: "73c0d747796c4975b83b7648a8ae6abd" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35487 } } peers { permanent_uuid: "0ff57111a35f4adaa4a5fb98b4d7dece" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41849 } }
I20260929 10:59:13.663316 28150 log.cc:824] T 9929a9b125de4b04948771f268534d20 P 0ff57111a35f4adaa4a5fb98b4d7dece: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:13.663594 28151 ts_tablet_manager.cc:1428] T 9929a9b125de4b04948771f268534d20 P 73c0d747796c4975b83b7648a8ae6abd: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 10:59:13.663668 27868 heartbeater.cc:503] Master 127.26.120.254:34281 was elected leader, sending a full tablet report...
I20260929 10:59:13.664103 28150 tablet_bootstrap.cc:492] T 9929a9b125de4b04948771f268534d20 P 0ff57111a35f4adaa4a5fb98b4d7dece: No bootstrap required, opened a new log
I20260929 10:59:13.664197 28150 ts_tablet_manager.cc:1397] T 9929a9b125de4b04948771f268534d20 P 0ff57111a35f4adaa4a5fb98b4d7dece: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260929 10:59:13.664309 28152 tablet_bootstrap.cc:492] T 9929a9b125de4b04948771f268534d20 P 23f0f05beebe49ed80beab70ee08c4ee: No bootstrap required, opened a new log
I20260929 10:59:13.664399 28152 ts_tablet_manager.cc:1397] T 9929a9b125de4b04948771f268534d20 P 23f0f05beebe49ed80beab70ee08c4ee: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:59:13.665828 28150 raft_consensus.cc:348] T 9929a9b125de4b04948771f268534d20 P 0ff57111a35f4adaa4a5fb98b4d7dece [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23f0f05beebe49ed80beab70ee08c4ee" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35459 } } peers { permanent_uuid: "73c0d747796c4975b83b7648a8ae6abd" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35487 } } peers { permanent_uuid: "0ff57111a35f4adaa4a5fb98b4d7dece" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41849 } }
I20260929 10:59:13.666002 28150 raft_consensus.cc:374] T 9929a9b125de4b04948771f268534d20 P 0ff57111a35f4adaa4a5fb98b4d7dece [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:13.666047 28150 raft_consensus.cc:729] T 9929a9b125de4b04948771f268534d20 P 0ff57111a35f4adaa4a5fb98b4d7dece [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ff57111a35f4adaa4a5fb98b4d7dece, State: Initialized, Role: FOLLOWER
I20260929 10:59:13.665940 28152 raft_consensus.cc:348] T 9929a9b125de4b04948771f268534d20 P 23f0f05beebe49ed80beab70ee08c4ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23f0f05beebe49ed80beab70ee08c4ee" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35459 } } peers { permanent_uuid: "73c0d747796c4975b83b7648a8ae6abd" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35487 } } peers { permanent_uuid: "0ff57111a35f4adaa4a5fb98b4d7dece" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41849 } }
I20260929 10:59:13.666198 28150 consensus_queue.cc:260] T 9929a9b125de4b04948771f268534d20 P 0ff57111a35f4adaa4a5fb98b4d7dece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23f0f05beebe49ed80beab70ee08c4ee" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35459 } } peers { permanent_uuid: "73c0d747796c4975b83b7648a8ae6abd" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35487 } } peers { permanent_uuid: "0ff57111a35f4adaa4a5fb98b4d7dece" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41849 } }
I20260929 10:59:13.666229 28152 raft_consensus.cc:374] T 9929a9b125de4b04948771f268534d20 P 23f0f05beebe49ed80beab70ee08c4ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:13.666281 28152 raft_consensus.cc:729] T 9929a9b125de4b04948771f268534d20 P 23f0f05beebe49ed80beab70ee08c4ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23f0f05beebe49ed80beab70ee08c4ee, State: Initialized, Role: FOLLOWER
I20260929 10:59:13.666386 28152 consensus_queue.cc:260] T 9929a9b125de4b04948771f268534d20 P 23f0f05beebe49ed80beab70ee08c4ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23f0f05beebe49ed80beab70ee08c4ee" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35459 } } peers { permanent_uuid: "73c0d747796c4975b83b7648a8ae6abd" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35487 } } peers { permanent_uuid: "0ff57111a35f4adaa4a5fb98b4d7dece" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41849 } }
I20260929 10:59:13.666486 28150 ts_tablet_manager.cc:1428] T 9929a9b125de4b04948771f268534d20 P 0ff57111a35f4adaa4a5fb98b4d7dece: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260929 10:59:13.666535 28002 heartbeater.cc:503] Master 127.26.120.254:34281 was elected leader, sending a full tablet report...
I20260929 10:59:13.666652 28152 ts_tablet_manager.cc:1428] T 9929a9b125de4b04948771f268534d20 P 23f0f05beebe49ed80beab70ee08c4ee: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 10:59:13.666764 28136 heartbeater.cc:503] Master 127.26.120.254:34281 was elected leader, sending a full tablet report...
I20260929 10:59:13.697744 28158 raft_consensus.cc:482] T 9929a9b125de4b04948771f268534d20 P 23f0f05beebe49ed80beab70ee08c4ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:59:13.697860 28158 raft_consensus.cc:504] T 9929a9b125de4b04948771f268534d20 P 23f0f05beebe49ed80beab70ee08c4ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23f0f05beebe49ed80beab70ee08c4ee" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35459 } } peers { permanent_uuid: "73c0d747796c4975b83b7648a8ae6abd" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35487 } } peers { permanent_uuid: "0ff57111a35f4adaa4a5fb98b4d7dece" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41849 } }
I20260929 10:59:13.698278 28158 leader_election.cc:288] T 9929a9b125de4b04948771f268534d20 P 23f0f05beebe49ed80beab70ee08c4ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73c0d747796c4975b83b7648a8ae6abd (127.26.120.193:35487), 0ff57111a35f4adaa4a5fb98b4d7dece (127.26.120.194:41849)
I20260929 10:59:13.702262 28148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:13.698560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:59607 (local address 127.26.120.194:41849)
0929 10:59:13.698602 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:13.698604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:13.698761 (+ 157us) server_negotiation.cc:407] Connection header received
0929 10:59:13.698796 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:13.698798 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:13.698806 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:13.698853 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:13.699016 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.699711 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.701293 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.701875 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.701955 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:13.701963 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:13.702145 (+ 182us) server_negotiation.cc:299] Negotiation successful
0929 10:59:13.702197 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 10:59:13.702272 28149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:13.698531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:44989 (local address 127.26.120.193:35487)
0929 10:59:13.698595 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:13.698597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:13.698606 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:59:13.698635 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:13.698636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:13.698645 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:13.698713 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:13.698910 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.699692 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.701293 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.701879 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.701954 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:13.701963 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:13.702145 (+ 182us) server_negotiation.cc:299] Negotiation successful
0929 10:59:13.702188 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 10:59:13.702308 28138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:13.698440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:35487 (local address 127.26.120.195:44989)
0929 10:59:13.698510 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:13.698523 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:13.698566 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:13.698726 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:13.698727 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:13.698729 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:13.698813 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:13.698815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.699783 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.699784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:13.701194 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:13.701197 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.701962 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.701963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:13.702047 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:13.702051 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:59:13.702130 (+ 79us) client_negotiation.cc:240] Negotiation successful
0929 10:59:13.702179 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":123}
I20260929 10:59:13.702317 28159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:13.698495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:41849 (local address 127.26.120.195:59607)
0929 10:59:13.698668 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:13.698675 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:13.698739 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:13.698877 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:13.698878 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:13.698879 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:13.698938 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:13.698939 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.699783 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.699784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:13.701194 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:13.701197 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.701962 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.701962 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:13.702047 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:13.702073 (+ 26us) client_negotiation.cc:769] Sending connection context
0929 10:59:13.702130 (+ 57us) client_negotiation.cc:240] Negotiation successful
0929 10:59:13.702179 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":85,"thread_start_us":118,"threads_started":1}
I20260929 10:59:13.702796 27814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9929a9b125de4b04948771f268534d20" candidate_uuid: "23f0f05beebe49ed80beab70ee08c4ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73c0d747796c4975b83b7648a8ae6abd" is_pre_election: true
I20260929 10:59:13.702982 27814 raft_consensus.cc:2454] T 9929a9b125de4b04948771f268534d20 P 73c0d747796c4975b83b7648a8ae6abd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23f0f05beebe49ed80beab70ee08c4ee in term 0.
I20260929 10:59:13.703258 28025 leader_election.cc:302] T 9929a9b125de4b04948771f268534d20 P 23f0f05beebe49ed80beab70ee08c4ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23f0f05beebe49ed80beab70ee08c4ee, 73c0d747796c4975b83b7648a8ae6abd; no voters:
I20260929 10:59:13.703419 28158 raft_consensus.cc:2781] T 9929a9b125de4b04948771f268534d20 P 23f0f05beebe49ed80beab70ee08c4ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:59:13.703504 28158 raft_consensus.cc:482] T 9929a9b125de4b04948771f268534d20 P 23f0f05beebe49ed80beab70ee08c4ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:59:13.703529 28158 raft_consensus.cc:3037] T 9929a9b125de4b04948771f268534d20 P 23f0f05beebe49ed80beab70ee08c4ee [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:13.704248 28158 raft_consensus.cc:504] T 9929a9b125de4b04948771f268534d20 P 23f0f05beebe49ed80beab70ee08c4ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23f0f05beebe49ed80beab70ee08c4ee" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35459 } } peers { permanent_uuid: "73c0d747796c4975b83b7648a8ae6abd" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35487 } } peers { permanent_uuid: "0ff57111a35f4adaa4a5fb98b4d7dece" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41849 } }
I20260929 10:59:13.704394 28158 leader_election.cc:288] T 9929a9b125de4b04948771f268534d20 P 23f0f05beebe49ed80beab70ee08c4ee [CANDIDATE]: Term 1 election: Requested vote from peers 73c0d747796c4975b83b7648a8ae6abd (127.26.120.193:35487), 0ff57111a35f4adaa4a5fb98b4d7dece (127.26.120.194:41849)
I20260929 10:59:13.702800 27956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9929a9b125de4b04948771f268534d20" candidate_uuid: "23f0f05beebe49ed80beab70ee08c4ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ff57111a35f4adaa4a5fb98b4d7dece" is_pre_election: true
I20260929 10:59:13.704571 27956 raft_consensus.cc:2454] T 9929a9b125de4b04948771f268534d20 P 0ff57111a35f4adaa4a5fb98b4d7dece [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23f0f05beebe49ed80beab70ee08c4ee in term 0.
I20260929 10:59:13.704617 27814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9929a9b125de4b04948771f268534d20" candidate_uuid: "23f0f05beebe49ed80beab70ee08c4ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73c0d747796c4975b83b7648a8ae6abd"
I20260929 10:59:13.704639 27955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9929a9b125de4b04948771f268534d20" candidate_uuid: "23f0f05beebe49ed80beab70ee08c4ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ff57111a35f4adaa4a5fb98b4d7dece"
I20260929 10:59:13.704692 27814 raft_consensus.cc:3037] T 9929a9b125de4b04948771f268534d20 P 73c0d747796c4975b83b7648a8ae6abd [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:13.704720 27955 raft_consensus.cc:3037] T 9929a9b125de4b04948771f268534d20 P 0ff57111a35f4adaa4a5fb98b4d7dece [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:13.705582 27955 raft_consensus.cc:2454] T 9929a9b125de4b04948771f268534d20 P 0ff57111a35f4adaa4a5fb98b4d7dece [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23f0f05beebe49ed80beab70ee08c4ee in term 1.
I20260929 10:59:13.705582 27814 raft_consensus.cc:2454] T 9929a9b125de4b04948771f268534d20 P 73c0d747796c4975b83b7648a8ae6abd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23f0f05beebe49ed80beab70ee08c4ee in term 1.
I20260929 10:59:13.705768 28025 leader_election.cc:302] T 9929a9b125de4b04948771f268534d20 P 23f0f05beebe49ed80beab70ee08c4ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23f0f05beebe49ed80beab70ee08c4ee, 73c0d747796c4975b83b7648a8ae6abd; no voters:
I20260929 10:59:13.705899 28158 raft_consensus.cc:2781] T 9929a9b125de4b04948771f268534d20 P 23f0f05beebe49ed80beab70ee08c4ee [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:59:13.706101 28158 raft_consensus.cc:686] T 9929a9b125de4b04948771f268534d20 P 23f0f05beebe49ed80beab70ee08c4ee [term 1 LEADER]: Becoming Leader. State: Replica: 23f0f05beebe49ed80beab70ee08c4ee, State: Running, Role: LEADER
I20260929 10:59:13.706205 28158 consensus_queue.cc:237] T 9929a9b125de4b04948771f268534d20 P 23f0f05beebe49ed80beab70ee08c4ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23f0f05beebe49ed80beab70ee08c4ee" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35459 } } peers { permanent_uuid: "73c0d747796c4975b83b7648a8ae6abd" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35487 } } peers { permanent_uuid: "0ff57111a35f4adaa4a5fb98b4d7dece" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41849 } }
I20260929 10:59:13.707165 27675 catalog_manager.cc:5495] T 9929a9b125de4b04948771f268534d20 P 23f0f05beebe49ed80beab70ee08c4ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 23f0f05beebe49ed80beab70ee08c4ee (127.26.120.195). New cstate: current_term: 1 leader_uuid: "23f0f05beebe49ed80beab70ee08c4ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23f0f05beebe49ed80beab70ee08c4ee" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73c0d747796c4975b83b7648a8ae6abd" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ff57111a35f4adaa4a5fb98b4d7dece" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41849 } health_report { overall_health: UNKNOWN } } }
Sep 29 10:59:13 dist-test-slave-2lzq krb5kdc[27632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679552, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.193@KRBTEST.COM
I20260929 10:59:13.716234 28149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:13.708870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34306 (local address 127.26.120.193:35487)
0929 10:59:13.708907 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:13.708909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:13.709070 (+ 161us) server_negotiation.cc:407] Connection header received
0929 10:59:13.709146 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:13.709147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:13.709157 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:13.709224 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:13.709773 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.710316 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.711058 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.711249 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.711337 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:13.713609 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:13.713621 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:13.713621 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:13.713628 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:13.715035 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:13.715437 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:13.715440 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:13.715441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:13.715493 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:13.715695 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:13.715695 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:13.715696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:13.715837 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:13.715992 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:13.716055 (+ 63us) server_negotiation.cc:299] Negotiation successful
0929 10:59:13.716132 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260929 10:59:13.718695 28003 tablet.cc:2369] T 9929a9b125de4b04948771f268534d20 P 0ff57111a35f4adaa4a5fb98b4d7dece: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 29 10:59:13 dist-test-slave-2lzq krb5kdc[27632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679553, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 10:59:13 dist-test-slave-2lzq krb5kdc[27632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679553, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.120.193@KRBTEST.COM
I20260929 10:59:13.739593 28149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:13.732238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34316 (local address 127.26.120.193:35487)
0929 10:59:13.732298 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:13.732300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:13.732374 (+ 74us) server_negotiation.cc:407] Connection header received
0929 10:59:13.732453 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:13.732454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:13.732463 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:13.732506 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:13.733130 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.733672 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.734423 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.734603 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.734657 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:13.736839 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:13.736841 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:13.736841 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:13.736848 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:13.738235 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:13.738734 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:13.738735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:13.738736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:13.738795 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:13.739014 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:13.739015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:13.739015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:13.739117 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:13.739219 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:13.739413 (+ 194us) server_negotiation.cc:299] Negotiation successful
0929 10:59:13.739489 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
Sep 29 10:59:13 dist-test-slave-2lzq krb5kdc[27632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 10:59:13 dist-test-slave-2lzq krb5kdc[27632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.193@KRBTEST.COM
I20260929 10:59:13.761278 28149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:13.754818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34330 (local address 127.26.120.193:35487)
0929 10:59:13.754893 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:13.754895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:13.754908 (+ 13us) server_negotiation.cc:407] Connection header received
0929 10:59:13.755005 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:13.755007 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:13.755038 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:13.755088 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:13.755622 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.756090 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.756794 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:13.756967 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:13.757015 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:13.759179 (+ 2164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:13.759181 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:13.759181 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:13.759188 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:13.760380 (+ 1192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:13.760680 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:13.760681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:13.760681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:13.760728 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:13.760890 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:13.760891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:13.760891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:13.760976 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:13.761057 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:13.761134 (+ 77us) server_negotiation.cc:299] Negotiation successful
0929 10:59:13.761186 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260929 10:59:13.761864 27782 tablet_service.cc:2908] Rejecting scan request for tablet 9929a9b125de4b04948771f268534d20: Uninitialized: clean time has not yet been initialized
I20260929 10:59:13.762746 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 27739
I20260929 10:59:13.767822 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 27873
I20260929 10:59:13.772521 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 28007
I20260929 10:59:13.777175 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 27645
2026-09-29T10:59:13Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (1915 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:59:13 dist-test-slave-2lzq krb5kdc[28174](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:59:13 dist-test-slave-2lzq krb5kdc[28174](info): set up 2 sockets
Sep 29 10:59:13 dist-test-slave-2lzq krb5kdc[28174](info): commencing operation
krb5kdc: starting...
W20260929 10:59:14.809945 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 10:59:14 dist-test-slave-2lzq krb5kdc[28174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679554, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:59:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:59:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.120.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
I20260929 10:59:14.927223 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:46489
--webserver_interface=127.26.120.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:38615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:46489
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:15.007786 28187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:15.007993 28187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:15.008016 28187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:15.009106 28187 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:59:15.009138 28187 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 10:59:15.009151 28187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:15.009164 28187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:15.009176 28187 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:59:15.009187 28187 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:59:15.010648 28187 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:38615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:46489
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:46489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:15.010838 28187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:15.011021 28187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:15.015843 28195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:15.015816 28192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:15.015825 28193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:15.016172 28187 server_base.cc:1034] running on GCE node
I20260929 10:59:15.016361 28187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:15.016599 28187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:15.017737 28187 hybrid_clock.cc:648] HybridClock initialized: now 1790679555017724 us; error 27 us; skew 500 ppm
Sep 29 10:59:15 dist-test-slave-2lzq krb5kdc[28174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679555, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:15.020323 28187 init.cc:377] Logged in from keytab as kudu/127.26.120.254@KRBTEST.COM (short username kudu)
I20260929 10:59:15.020813 28187 webserver.cc:460] Webserver started at http://127.26.120.254:44199/ using document root <none> and password file <none>
I20260929 10:59:15.021061 28187 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:15.021108 28187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:15.021265 28187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:15.022421 28187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "5c435fdf336a4883b3edbc6b75ee9e73"
format_stamp: "Formatted at 2026-09-29 10:59:15 on dist-test-slave-2lzq"
server_key: "84d1a031149eec374978dc3bef9eb0e3"
server_key_iv: "170b486da79fc74adb41bab73e089687"
server_key_version: "encryptionkey@0"
I20260929 10:59:15.022797 28187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "5c435fdf336a4883b3edbc6b75ee9e73"
format_stamp: "Formatted at 2026-09-29 10:59:15 on dist-test-slave-2lzq"
server_key: "84d1a031149eec374978dc3bef9eb0e3"
server_key_iv: "170b486da79fc74adb41bab73e089687"
server_key_version: "encryptionkey@0"
I20260929 10:59:15.024161 28187 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:59:15.024962 28202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:15.025247 28187 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:59:15.025355 28187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "5c435fdf336a4883b3edbc6b75ee9e73"
format_stamp: "Formatted at 2026-09-29 10:59:15 on dist-test-slave-2lzq"
server_key: "84d1a031149eec374978dc3bef9eb0e3"
server_key_iv: "170b486da79fc74adb41bab73e089687"
server_key_version: "encryptionkey@0"
I20260929 10:59:15.025432 28187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:15.036051 28187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:15.038450 28187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:15.038628 28187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:15.043695 28187 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:46489
I20260929 10:59:15.043756 28264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:46489 every 8 connection(s)
I20260929 10:59:15.044072 28187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 10:59:15.044772 28265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:15.047298 28265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c435fdf336a4883b3edbc6b75ee9e73: Bootstrap starting.
I20260929 10:59:15.047938 28265 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c435fdf336a4883b3edbc6b75ee9e73: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:15.048247 28265 log.cc:824] T 00000000000000000000000000000000 P 5c435fdf336a4883b3edbc6b75ee9e73: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:15.049015 28265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c435fdf336a4883b3edbc6b75ee9e73: No bootstrap required, opened a new log
I20260929 10:59:15.051234 28265 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c435fdf336a4883b3edbc6b75ee9e73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c435fdf336a4883b3edbc6b75ee9e73" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 46489 } }
I20260929 10:59:15.051407 28265 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c435fdf336a4883b3edbc6b75ee9e73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:15.051456 28265 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c435fdf336a4883b3edbc6b75ee9e73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c435fdf336a4883b3edbc6b75ee9e73, State: Initialized, Role: FOLLOWER
I20260929 10:59:15.051628 28265 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c435fdf336a4883b3edbc6b75ee9e73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c435fdf336a4883b3edbc6b75ee9e73" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 46489 } }
I20260929 10:59:15.051712 28265 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c435fdf336a4883b3edbc6b75ee9e73 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:59:15.051752 28265 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c435fdf336a4883b3edbc6b75ee9e73 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:59:15.051805 28265 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c435fdf336a4883b3edbc6b75ee9e73 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:15.052230 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 28187
I20260929 10:59:15.052304 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 10:59:15.052421 28265 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c435fdf336a4883b3edbc6b75ee9e73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c435fdf336a4883b3edbc6b75ee9e73" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 46489 } }
I20260929 10:59:15.052464 27107 external_mini_cluster.cc:1428] Setting key aefb8a1b3eb4c61d6352f611c5b49ac9
I20260929 10:59:15.052536 28265 leader_election.cc:302] T 00000000000000000000000000000000 P 5c435fdf336a4883b3edbc6b75ee9e73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c435fdf336a4883b3edbc6b75ee9e73; no voters:
I20260929 10:59:15.052726 28265 leader_election.cc:288] T 00000000000000000000000000000000 P 5c435fdf336a4883b3edbc6b75ee9e73 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:59:15.052757 28268 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c435fdf336a4883b3edbc6b75ee9e73 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:59:15.052970 28268 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c435fdf336a4883b3edbc6b75ee9e73 [term 1 LEADER]: Becoming Leader. State: Replica: 5c435fdf336a4883b3edbc6b75ee9e73, State: Running, Role: LEADER
I20260929 10:59:15.053004 28265 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c435fdf336a4883b3edbc6b75ee9e73 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:59:15.053117 28268 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c435fdf336a4883b3edbc6b75ee9e73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c435fdf336a4883b3edbc6b75ee9e73" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 46489 } }
I20260929 10:59:15.053881 28271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c435fdf336a4883b3edbc6b75ee9e73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c435fdf336a4883b3edbc6b75ee9e73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c435fdf336a4883b3edbc6b75ee9e73" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 46489 } } }
I20260929 10:59:15.053969 28271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c435fdf336a4883b3edbc6b75ee9e73 [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:15.054158 28269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c435fdf336a4883b3edbc6b75ee9e73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c435fdf336a4883b3edbc6b75ee9e73. Latest consensus state: current_term: 1 leader_uuid: "5c435fdf336a4883b3edbc6b75ee9e73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c435fdf336a4883b3edbc6b75ee9e73" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 46489 } } }
I20260929 10:59:15.054239 28269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c435fdf336a4883b3edbc6b75ee9e73 [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:15.054379 28280 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:59:15.054905 28280 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:59:15.056321 28280 catalog_manager.cc:1347] Generated new cluster ID: 1bcb0e64901a48efaaeb26c63bd05ba8
I20260929 10:59:15.056373 28280 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:59:15.060369 28280 catalog_manager.cc:1370] Generated new certificate authority record
Sep 29 10:59:15 dist-test-slave-2lzq krb5kdc[28174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679554, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:15.060950 28280 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:59:15.064328 28280 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c435fdf336a4883b3edbc6b75ee9e73: Generated new TSK 0
I20260929 10:59:15.064494 28280 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:59:15.065088 28272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:15.052925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54736 (local address 127.26.120.254:46489)
0929 10:59:15.053122 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:15.053131 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:15.053398 (+ 267us) server_negotiation.cc:407] Connection header received
0929 10:59:15.053523 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:15.053530 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:15.053604 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:15.053743 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:15.054912 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.056321 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.057405 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.058011 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.058141 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:15.061076 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:15.061081 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:15.061082 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:15.061098 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:15.063347 (+ 2249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:15.063769 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:15.063773 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:15.063773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:15.063901 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:15.064133 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:15.064134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:15.064134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:15.064427 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:15.064582 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:15.064823 (+ 241us) server_negotiation.cc:299] Negotiation successful
0929 10:59:15.064945 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":73,"threads_started":1}
WARNING: no policy specified for kudu/127.26.120.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab.
Entry for principal kudu/127.26.120.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab.
I20260929 10:59:15.093559 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.193:0
--local_ip_for_outbound_sockets=127.26.120.193
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:46489
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:38615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:15.182025 28291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:15.182281 28291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:15.182307 28291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:15.183480 28291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:15.183521 28291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:15.183540 28291 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 10:59:15.183558 28291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.193
I20260929 10:59:15.184966 28291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:38615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.120.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.120.254:46489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.193
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:15.185195 28291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:15.185439 28291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 10:59:15.190339 28291 server_base.cc:1034] running on GCE node
W20260929 10:59:15.190315 28296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:15.190322 28297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:15.190320 28299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:15.190773 28291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:15.191056 28291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:15.192230 28291 hybrid_clock.cc:648] HybridClock initialized: now 1790679555192210 us; error 35 us; skew 500 ppm
Sep 29 10:59:15 dist-test-slave-2lzq krb5kdc[28174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679555, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:15.194748 28291 init.cc:377] Logged in from keytab as kudu/127.26.120.193@KRBTEST.COM (short username kudu)
I20260929 10:59:15.195164 28291 webserver.cc:460] Webserver started at http://127.26.120.193:43425/ using document root <none> and password file <none>
I20260929 10:59:15.195386 28291 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:15.195456 28291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:15.195604 28291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:15.196614 28291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-0/data/instance:
uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6"
format_stamp: "Formatted at 2026-09-29 10:59:15 on dist-test-slave-2lzq"
server_key: "b81d8df742ff4951cac348ff2d8a7bd2"
server_key_iv: "34680d7ad724de5b4fb802cb6c64111e"
server_key_version: "encryptionkey@0"
I20260929 10:59:15.196970 28291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance:
uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6"
format_stamp: "Formatted at 2026-09-29 10:59:15 on dist-test-slave-2lzq"
server_key: "b81d8df742ff4951cac348ff2d8a7bd2"
server_key_iv: "34680d7ad724de5b4fb802cb6c64111e"
server_key_version: "encryptionkey@0"
I20260929 10:59:15.198349 28291 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 10:59:15.199172 28306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:15.199335 28291 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:59:15.199433 28291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-0/wal
uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6"
format_stamp: "Formatted at 2026-09-29 10:59:15 on dist-test-slave-2lzq"
server_key: "b81d8df742ff4951cac348ff2d8a7bd2"
server_key_iv: "34680d7ad724de5b4fb802cb6c64111e"
server_key_version: "encryptionkey@0"
I20260929 10:59:15.199514 28291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:15.207544 28291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:15.209733 28291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:15.209896 28291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:15.210403 28291 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:15.210465 28291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:15.210496 28291 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:15.210523 28291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:15.217398 28291 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.193:35425
I20260929 10:59:15.217490 28421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.193:35425 every 8 connection(s)
Sep 29 10:59:15 dist-test-slave-2lzq krb5kdc[28174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679555, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:15.217801 28291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
I20260929 10:59:15.218927 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 28291
I20260929 10:59:15.219039 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance
I20260929 10:59:15.219204 27107 external_mini_cluster.cc:1428] Setting key 9237a7dd68d5637be0e962d507a051f8
I20260929 10:59:15.221125 28272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:15.210857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:39637 (local address 127.26.120.254:46489)
0929 10:59:15.210903 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:15.210905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:15.211129 (+ 224us) server_negotiation.cc:407] Connection header received
0929 10:59:15.211612 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:15.211613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:15.211625 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:15.211688 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:15.212629 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.213448 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.215133 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.215476 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.215533 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:15.218319 (+ 2786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:15.218320 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:15.218321 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:15.218327 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:15.219749 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:15.220150 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:15.220151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:15.220151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:15.220214 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:15.220443 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:15.220443 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:15.220444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:15.220587 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:15.220657 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:15.220887 (+ 230us) server_negotiation.cc:299] Negotiation successful
0929 10:59:15.220983 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 10:59:15.221206 28315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:15.210859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:46489 (local address 127.26.120.193:39637)
0929 10:59:15.211018 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:15.211051 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:15.211509 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:15.211806 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:15.211811 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:15.212204 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:15.212521 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:15.212527 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.213551 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.213552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:15.215021 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:15.215024 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.215563 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.215564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:15.215718 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:15.216303 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:15.216325 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:15.218230 (+ 1905us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:15.219829 (+ 1599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:15.219836 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:15.219839 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:15.220076 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:15.220283 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:15.220284 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:15.220284 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:15.220378 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:15.220652 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:15.220657 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:15.220763 (+ 106us) client_negotiation.cc:769] Sending connection context
0929 10:59:15.220844 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 10:59:15.220967 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":79,"threads_started":1}
I20260929 10:59:15.221709 28422 heartbeater.cc:347] Connected to a master server at 127.26.120.254:46489
I20260929 10:59:15.221793 28422 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:15.221980 28422 heartbeater.cc:511] Master 127.26.120.254:46489 requested a full tablet report, sending...
I20260929 10:59:15.222532 28219 ts_manager.cc:194] Registered new tserver with Master: 4dbb8f2d2c1d4e9e8c467c2301e71ba6 (127.26.120.193:35425)
I20260929 10:59:15.223357 28219 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.193@KRBTEST.COM'} at 127.26.120.193:39637
WARNING: no policy specified for kudu/127.26.120.194@KRBTEST.COM; defaulting to no policy
I20260929 10:59:15.229079 28429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:15.222864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:46489 (local address 127.26.120.193:46343)
0929 10:59:15.223009 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:15.223020 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:15.223081 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:15.223202 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:15.223203 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:15.223403 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:15.223487 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:15.223489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.224500 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.224501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:15.224981 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:15.224982 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.225497 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.225498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:15.225552 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:15.225993 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:15.225997 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:15.226318 (+ 321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:15.228011 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:15.228012 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:15.228012 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:15.228219 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:15.228427 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:15.228427 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:15.228428 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:15.228483 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:15.228725 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:15.228726 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:15.228789 (+ 63us) client_negotiation.cc:769] Sending connection context
0929 10:59:15.228881 (+ 92us) client_negotiation.cc:240] Negotiation successful
0929 10:59:15.228981 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":72,"threads_started":1}
I20260929 10:59:15.229089 28272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:15.222797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:46343 (local address 127.26.120.254:46489)
0929 10:59:15.222834 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:15.222836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:15.223091 (+ 255us) server_negotiation.cc:407] Connection header received
0929 10:59:15.223121 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:15.223121 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:15.223132 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:15.223190 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:15.223595 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.224391 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.225067 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.225413 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.225491 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:15.226405 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:15.226407 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:15.226407 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:15.226415 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:15.227931 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:15.228288 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:15.228289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:15.228289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:15.228359 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:15.228544 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:15.228545 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:15.228545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:15.228667 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:15.228733 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:15.228910 (+ 177us) server_negotiation.cc:299] Negotiation successful
0929 10:59:15.229009 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab.
Entry for principal kudu/127.26.120.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab.
I20260929 10:59:15.245199 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.194:0
--local_ip_for_outbound_sockets=127.26.120.194
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:46489
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:38615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:15.325161 28431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:15.325395 28431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:15.325438 28431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:15.326673 28431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:15.326737 28431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:15.326769 28431 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 10:59:15.326810 28431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.194
I20260929 10:59:15.328310 28431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:38615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.120.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.120.254:46489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.194
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:15.328593 28431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:15.328821 28431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:15.333827 28439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:15.333860 28437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:15.333863 28436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:15.334105 28431 server_base.cc:1034] running on GCE node
I20260929 10:59:15.334331 28431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:15.334581 28431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:15.335719 28431 hybrid_clock.cc:648] HybridClock initialized: now 1790679555335695 us; error 36 us; skew 500 ppm
Sep 29 10:59:15 dist-test-slave-2lzq krb5kdc[28174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679555, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:15.338193 28431 init.cc:377] Logged in from keytab as kudu/127.26.120.194@KRBTEST.COM (short username kudu)
I20260929 10:59:15.338675 28431 webserver.cc:460] Webserver started at http://127.26.120.194:33409/ using document root <none> and password file <none>
I20260929 10:59:15.338924 28431 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:15.338992 28431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:15.339134 28431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:15.340188 28431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-1/data/instance:
uuid: "571b683415fe42e194f078c902ed0f10"
format_stamp: "Formatted at 2026-09-29 10:59:15 on dist-test-slave-2lzq"
server_key: "842ea9d0ba3ba47ec5a208a93f3d0d66"
server_key_iv: "bb2fb5faa7bd3f0414600953e93a646b"
server_key_version: "encryptionkey@0"
I20260929 10:59:15.340560 28431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance:
uuid: "571b683415fe42e194f078c902ed0f10"
format_stamp: "Formatted at 2026-09-29 10:59:15 on dist-test-slave-2lzq"
server_key: "842ea9d0ba3ba47ec5a208a93f3d0d66"
server_key_iv: "bb2fb5faa7bd3f0414600953e93a646b"
server_key_version: "encryptionkey@0"
I20260929 10:59:15.341881 28431 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 10:59:15.342744 28446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:15.342921 28431 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:59:15.343036 28431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-1/wal
uuid: "571b683415fe42e194f078c902ed0f10"
format_stamp: "Formatted at 2026-09-29 10:59:15 on dist-test-slave-2lzq"
server_key: "842ea9d0ba3ba47ec5a208a93f3d0d66"
server_key_iv: "bb2fb5faa7bd3f0414600953e93a646b"
server_key_version: "encryptionkey@0"
I20260929 10:59:15.343124 28431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:15.368681 28431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:15.370769 28431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:15.370934 28431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:15.371547 28431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:15.371598 28431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:15.371631 28431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:15.371654 28431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 10:59:15 dist-test-slave-2lzq krb5kdc[28174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679555, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:15.378786 28431 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.194:37997
I20260929 10:59:15.378880 28561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.194:37997 every 8 connection(s)
I20260929 10:59:15.379189 28431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
I20260929 10:59:15.379842 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 28431
I20260929 10:59:15.379913 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance
I20260929 10:59:15.380093 27107 external_mini_cluster.cc:1428] Setting key ae0483fa90118e54ef8822831517274c
I20260929 10:59:15.381729 28272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:15.371999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:34103 (local address 127.26.120.254:46489)
0929 10:59:15.372057 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:15.372059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:15.372320 (+ 261us) server_negotiation.cc:407] Connection header received
0929 10:59:15.373068 (+ 748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:15.373070 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:15.373081 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:15.373138 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:15.374076 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.374588 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.375608 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.375834 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.375911 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:15.378589 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:15.378591 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:15.378591 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:15.378599 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:15.380111 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:15.380603 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:15.380604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:15.380605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:15.380670 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:15.380947 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:15.380948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:15.380948 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:15.381072 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:15.381150 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:15.381434 (+ 284us) server_negotiation.cc:299] Negotiation successful
0929 10:59:15.381627 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260929 10:59:15.381903 28455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:15.371974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:46489 (local address 127.26.120.194:34103)
0929 10:59:15.372165 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:15.372218 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:15.372923 (+ 705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:15.373225 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:15.373229 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:15.373641 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:15.373949 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:15.373955 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.374693 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.374695 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:15.375511 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:15.375513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.375918 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.375919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:15.376043 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:15.376545 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:15.376552 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:15.378479 (+ 1927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:15.380203 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:15.380209 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:15.380212 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:15.380518 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:15.380737 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:15.380738 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:15.380738 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:15.380867 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:15.381141 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:15.381147 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:15.381293 (+ 146us) client_negotiation.cc:769] Sending connection context
0929 10:59:15.381399 (+ 106us) client_negotiation.cc:240] Negotiation successful
0929 10:59:15.381640 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":89,"threads_started":1}
I20260929 10:59:15.382498 28562 heartbeater.cc:347] Connected to a master server at 127.26.120.254:46489
I20260929 10:59:15.382587 28562 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:15.382772 28562 heartbeater.cc:511] Master 127.26.120.254:46489 requested a full tablet report, sending...
I20260929 10:59:15.383109 28219 ts_manager.cc:194] Registered new tserver with Master: 571b683415fe42e194f078c902ed0f10 (127.26.120.194:37997)
I20260929 10:59:15.383616 28219 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.194@KRBTEST.COM'} at 127.26.120.194:34103
WARNING: no policy specified for kudu/127.26.120.195@KRBTEST.COM; defaulting to no policy
I20260929 10:59:15.389247 28272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:15.383580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:47945 (local address 127.26.120.254:46489)
0929 10:59:15.383617 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:15.383619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:15.383768 (+ 149us) server_negotiation.cc:407] Connection header received
0929 10:59:15.383816 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:15.383817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:15.383828 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:15.383884 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:15.384232 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.384691 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.385426 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.385629 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.385713 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:15.386658 (+ 945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:15.386660 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:15.386660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:15.386667 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:15.388026 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:15.388383 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:15.388384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:15.388385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:15.388442 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:15.388653 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:15.388654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:15.388654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:15.388838 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:15.388915 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:15.389084 (+ 169us) server_negotiation.cc:299] Negotiation successful
0929 10:59:15.389153 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 10:59:15.389261 28569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:15.383525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:46489 (local address 127.26.120.194:47945)
0929 10:59:15.383681 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:15.383692 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:15.383760 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:15.383888 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:15.383889 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:15.384086 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:15.384147 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:15.384149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.384791 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.384792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:15.385320 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:15.385321 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.385716 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.385717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:15.385791 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:15.386255 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:15.386259 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:15.386567 (+ 308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:15.388128 (+ 1561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:15.388128 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:15.388129 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:15.388308 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:15.388523 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:15.388524 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:15.388524 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:15.388588 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:15.388917 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:15.388919 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:15.388966 (+ 47us) client_negotiation.cc:769] Sending connection context
0929 10:59:15.389047 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 10:59:15.389154 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":60,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab.
Entry for principal kudu/127.26.120.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab.
I20260929 10:59:15.405404 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.195:0
--local_ip_for_outbound_sockets=127.26.120.195
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:46489
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:38615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:15.490212 28571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:15.490417 28571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:15.490440 28571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:15.491612 28571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:15.491652 28571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:15.491673 28571 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 10:59:15.491690 28571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.195
I20260929 10:59:15.493152 28571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:38615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.120.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.120.254:46489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.195
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:15.493392 28571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:15.493611 28571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 10:59:15.498550 28571 server_base.cc:1034] running on GCE node
W20260929 10:59:15.498557 28579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:15.498713 28576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:15.498553 28577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:15.498932 28571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:15.499125 28571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:15.500283 28571 hybrid_clock.cc:648] HybridClock initialized: now 1790679555500265 us; error 33 us; skew 500 ppm
Sep 29 10:59:15 dist-test-slave-2lzq krb5kdc[28174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679555, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:15.502813 28571 init.cc:377] Logged in from keytab as kudu/127.26.120.195@KRBTEST.COM (short username kudu)
I20260929 10:59:15.503278 28571 webserver.cc:460] Webserver started at http://127.26.120.195:41045/ using document root <none> and password file <none>
I20260929 10:59:15.503512 28571 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:15.503587 28571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:15.503739 28571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:15.504766 28571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-2/data/instance:
uuid: "dda8b5ea355c47b6ab2884509a4c085e"
format_stamp: "Formatted at 2026-09-29 10:59:15 on dist-test-slave-2lzq"
server_key: "e699a635bed9a17914f3c10b9417b8d9"
server_key_iv: "ed14a9d0991c8046f76d21e7fceb85cb"
server_key_version: "encryptionkey@0"
I20260929 10:59:15.505126 28571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance:
uuid: "dda8b5ea355c47b6ab2884509a4c085e"
format_stamp: "Formatted at 2026-09-29 10:59:15 on dist-test-slave-2lzq"
server_key: "e699a635bed9a17914f3c10b9417b8d9"
server_key_iv: "ed14a9d0991c8046f76d21e7fceb85cb"
server_key_version: "encryptionkey@0"
I20260929 10:59:15.506492 28571 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 10:59:15.507203 28586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:15.507372 28571 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:59:15.507470 28571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-2/wal
uuid: "dda8b5ea355c47b6ab2884509a4c085e"
format_stamp: "Formatted at 2026-09-29 10:59:15 on dist-test-slave-2lzq"
server_key: "e699a635bed9a17914f3c10b9417b8d9"
server_key_iv: "ed14a9d0991c8046f76d21e7fceb85cb"
server_key_version: "encryptionkey@0"
I20260929 10:59:15.507553 28571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:15.529983 28571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:15.532366 28571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:15.532536 28571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:15.533131 28571 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:15.533181 28571 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:15.533213 28571 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:15.533234 28571 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 10:59:15 dist-test-slave-2lzq krb5kdc[28174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679555, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:15.540633 28571 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.195:43079
I20260929 10:59:15.540710 28701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.195:43079 every 8 connection(s)
I20260929 10:59:15.540997 28571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
I20260929 10:59:15.543331 28272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:15.533543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:53469 (local address 127.26.120.254:46489)
0929 10:59:15.533615 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:15.533618 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:15.533850 (+ 232us) server_negotiation.cc:407] Connection header received
0929 10:59:15.534414 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:15.534416 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:15.534430 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:15.534512 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:15.535065 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.535901 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.536956 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.537177 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.537258 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:15.540603 (+ 3345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:15.540605 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:15.540605 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:15.540614 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:15.542176 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:15.542584 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:15.542585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:15.542585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:15.542640 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:15.542842 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:15.542843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:15.542843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:15.542951 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:15.543002 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:15.543200 (+ 198us) server_negotiation.cc:299] Negotiation successful
0929 10:59:15.543265 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 10:59:15.543576 28595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:15.533547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:46489 (local address 127.26.120.195:53469)
0929 10:59:15.533714 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:15.533752 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:15.534317 (+ 565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:15.534576 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:15.534580 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:15.534808 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:15.534988 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:15.534992 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.536019 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.536020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:15.536859 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:15.536861 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.537266 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.537267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:15.537409 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:15.537971 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:15.537984 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:15.540484 (+ 2500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:15.542250 (+ 1766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:15.542257 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:15.542258 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:15.542514 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:15.542698 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:15.542698 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:15.542698 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:15.542791 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:15.542992 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:15.542994 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:15.543107 (+ 113us) client_negotiation.cc:769] Sending connection context
0929 10:59:15.543186 (+ 79us) client_negotiation.cc:240] Negotiation successful
0929 10:59:15.543305 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":90,"threads_started":1}
I20260929 10:59:15.544129 28702 heartbeater.cc:347] Connected to a master server at 127.26.120.254:46489
I20260929 10:59:15.544214 28702 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:15.544404 28702 heartbeater.cc:511] Master 127.26.120.254:46489 requested a full tablet report, sending...
I20260929 10:59:15.544775 28219 ts_manager.cc:194] Registered new tserver with Master: dda8b5ea355c47b6ab2884509a4c085e (127.26.120.195:43079)
I20260929 10:59:15.545248 28219 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.195@KRBTEST.COM'} at 127.26.120.195:53469
I20260929 10:59:15.550748 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 28571
I20260929 10:59:15.550845 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance
I20260929 10:59:15.550959 28708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:15.545239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:46489 (local address 127.26.120.195:55419)
0929 10:59:15.545395 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:15.545406 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:15.545461 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:15.545591 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:15.545591 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:15.545839 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:15.545906 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:15.545908 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.546696 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.546697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:15.547204 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:15.547205 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.547554 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.547554 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:15.547608 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:15.547973 (+ 365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:15.547978 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:15.548263 (+ 285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:15.549824 (+ 1561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:15.549825 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:15.549825 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:15.549981 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:15.550269 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:15.550270 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:15.550270 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:15.550329 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:15.550598 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:15.550599 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:15.550652 (+ 53us) client_negotiation.cc:769] Sending connection context
0929 10:59:15.550735 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 10:59:15.550844 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":70,"threads_started":1}
I20260929 10:59:15.550963 28272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:15.545308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:55419 (local address 127.26.120.254:46489)
0929 10:59:15.545386 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:15.545388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:15.545469 (+ 81us) server_negotiation.cc:407] Connection header received
0929 10:59:15.545518 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:15.545519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:15.545529 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:15.545576 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:15.545982 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.546611 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.547280 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.547484 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.547540 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:15.548346 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:15.548348 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:15.548348 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:15.548355 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:15.549752 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:15.550136 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:15.550137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:15.550137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:15.550201 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:15.550389 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:15.550390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:15.550391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:15.550506 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:15.550559 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:15.550760 (+ 201us) server_negotiation.cc:299] Negotiation successful
0929 10:59:15.550852 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 10:59:15.551033 27107 external_mini_cluster.cc:1428] Setting key ccb38c1f94f38b533ed9eb21be3d92f3
I20260929 10:59:15.551911 27107 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 10:59:15 dist-test-slave-2lzq krb5kdc[28174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679555, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 10:59:15 dist-test-slave-2lzq krb5kdc[28174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679555, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:15.574471 28272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:15.568036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36830 (local address 127.26.120.254:46489)
0929 10:59:15.568105 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:15.568107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:15.568116 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:59:15.568150 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:15.568151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:15.568160 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:15.568211 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:15.568733 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.569216 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.569915 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.570164 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.570216 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:15.572316 (+ 2100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:15.572317 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:15.572318 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:15.572324 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:15.573498 (+ 1174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:15.573789 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:15.573790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:15.573790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:15.573838 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:15.573994 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:15.573994 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:15.573994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:15.574150 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:15.574226 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:15.574297 (+ 71us) server_negotiation.cc:299] Negotiation successful
0929 10:59:15.574353 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 10:59:15.575913 28219 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36830:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:59:15.577358 28219 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:59:15.584007 28719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:15.579574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:37997 (local address 127.0.0.1:59460)
0929 10:59:15.579840 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:15.579868 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:15.579947 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:15.580451 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:15.580452 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:15.580454 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:15.580509 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:15.580511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.581334 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.581335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:15.582876 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:15.582879 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.583705 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.583707 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:15.583789 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:15.583803 (+ 14us) client_negotiation.cc:769] Sending connection context
0929 10:59:15.583871 (+ 68us) client_negotiation.cc:240] Negotiation successful
0929 10:59:15.583926 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":25,"thread_start_us":83,"threads_started":1}
I20260929 10:59:15.584108 28722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:15.579687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59460 (local address 127.26.120.194:37997)
0929 10:59:15.580252 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:15.580256 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:15.580264 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:59:15.580298 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:15.580301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:15.580316 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:15.580387 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:15.580567 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.581245 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.582962 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.583612 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.583696 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:15.583754 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:15.583944 (+ 190us) server_negotiation.cc:299] Negotiation successful
0929 10:59:15.583999 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":98,"threads_started":1}
I20260929 10:59:15.584195 28724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:15.579832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36678 (local address 127.26.120.195:43079)
0929 10:59:15.580011 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:15.580016 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:15.580123 (+ 107us) server_negotiation.cc:407] Connection header received
0929 10:59:15.580317 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:15.580320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:15.580338 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:15.580576 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:15.580632 (+ 56us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.581643 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.583017 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.583759 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.583834 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:15.583885 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:15.584048 (+ 163us) server_negotiation.cc:299] Negotiation successful
0929 10:59:15.584095 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":87,"threads_started":1}
I20260929 10:59:15.584394 28721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:15.579649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33560 (local address 127.26.120.193:35425)
0929 10:59:15.580031 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:15.580036 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:15.580045 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:59:15.580087 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:15.580090 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:15.580107 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:15.580186 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:15.580372 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.581555 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.582979 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.583764 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.583834 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:15.583885 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:15.584236 (+ 351us) server_negotiation.cc:299] Negotiation successful
0929 10:59:15.584284 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":70,"threads_started":1}
I20260929 10:59:15.584172 28720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:15.579532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:35425 (local address 127.0.0.1:33560)
0929 10:59:15.579818 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:15.579862 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:15.579935 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:15.580203 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:15.580206 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:15.580208 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:15.580277 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:15.580279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.581668 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.581668 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:15.582882 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:15.582884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.583921 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.583922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:15.583993 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:15.583997 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:59:15.584049 (+ 52us) client_negotiation.cc:240] Negotiation successful
0929 10:59:15.584093 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":61,"threads_started":1}
I20260929 10:59:15.584311 28723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:15.579740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:43079 (local address 127.0.0.1:36678)
0929 10:59:15.580076 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:15.580084 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:15.580134 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:15.580438 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:15.580440 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:15.580442 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:15.580495 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:15.580497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.581701 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.581702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:15.582946 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:15.582949 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.583830 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.583831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:15.583894 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:15.583896 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 10:59:15.583943 (+ 47us) client_negotiation.cc:240] Negotiation successful
0929 10:59:15.583974 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":15,"thread_start_us":55,"threads_started":1}
I20260929 10:59:15.585177 28496 tablet_service.cc:1467] Processing CreateTablet for tablet 932d03c8a10b468c9425b944a165c614 (DEFAULT_TABLE table=test-table [id=1f1fca2f86104b738398135d1abcf82f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:15.585330 28356 tablet_service.cc:1467] Processing CreateTablet for tablet 932d03c8a10b468c9425b944a165c614 (DEFAULT_TABLE table=test-table [id=1f1fca2f86104b738398135d1abcf82f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:15.585494 28496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 932d03c8a10b468c9425b944a165c614. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:15.585695 28356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 932d03c8a10b468c9425b944a165c614. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:15.588524 28725 tablet_bootstrap.cc:492] T 932d03c8a10b468c9425b944a165c614 P 571b683415fe42e194f078c902ed0f10: Bootstrap starting.
I20260929 10:59:15.588565 28726 tablet_bootstrap.cc:492] T 932d03c8a10b468c9425b944a165c614 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6: Bootstrap starting.
I20260929 10:59:15.588994 28622 tablet_service.cc:1467] Processing CreateTablet for tablet 932d03c8a10b468c9425b944a165c614 (DEFAULT_TABLE table=test-table [id=1f1fca2f86104b738398135d1abcf82f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:15.589134 28726 tablet_bootstrap.cc:654] T 932d03c8a10b468c9425b944a165c614 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:15.589259 28725 tablet_bootstrap.cc:654] T 932d03c8a10b468c9425b944a165c614 P 571b683415fe42e194f078c902ed0f10: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:15.589318 28622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 932d03c8a10b468c9425b944a165c614. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:15.589453 28726 log.cc:824] T 932d03c8a10b468c9425b944a165c614 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:15.589589 28725 log.cc:824] T 932d03c8a10b468c9425b944a165c614 P 571b683415fe42e194f078c902ed0f10: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:15.590333 28725 tablet_bootstrap.cc:492] T 932d03c8a10b468c9425b944a165c614 P 571b683415fe42e194f078c902ed0f10: No bootstrap required, opened a new log
I20260929 10:59:15.590405 28725 ts_tablet_manager.cc:1397] T 932d03c8a10b468c9425b944a165c614 P 571b683415fe42e194f078c902ed0f10: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:59:15.592468 28729 tablet_bootstrap.cc:492] T 932d03c8a10b468c9425b944a165c614 P dda8b5ea355c47b6ab2884509a4c085e: Bootstrap starting.
I20260929 10:59:15.592439 28725 raft_consensus.cc:348] T 932d03c8a10b468c9425b944a165c614 P 571b683415fe42e194f078c902ed0f10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } }
I20260929 10:59:15.592628 28725 raft_consensus.cc:374] T 932d03c8a10b468c9425b944a165c614 P 571b683415fe42e194f078c902ed0f10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:15.592679 28725 raft_consensus.cc:729] T 932d03c8a10b468c9425b944a165c614 P 571b683415fe42e194f078c902ed0f10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 571b683415fe42e194f078c902ed0f10, State: Initialized, Role: FOLLOWER
I20260929 10:59:15.592805 28725 consensus_queue.cc:260] T 932d03c8a10b468c9425b944a165c614 P 571b683415fe42e194f078c902ed0f10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } }
I20260929 10:59:15.593061 28725 ts_tablet_manager.cc:1428] T 932d03c8a10b468c9425b944a165c614 P 571b683415fe42e194f078c902ed0f10: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260929 10:59:15.593089 28726 tablet_bootstrap.cc:492] T 932d03c8a10b468c9425b944a165c614 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6: No bootstrap required, opened a new log
I20260929 10:59:15.593142 28726 ts_tablet_manager.cc:1397] T 932d03c8a10b468c9425b944a165c614 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260929 10:59:15.593176 28562 heartbeater.cc:503] Master 127.26.120.254:46489 was elected leader, sending a full tablet report...
I20260929 10:59:15.593567 28729 tablet_bootstrap.cc:654] T 932d03c8a10b468c9425b944a165c614 P dda8b5ea355c47b6ab2884509a4c085e: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:15.593921 28729 log.cc:824] T 932d03c8a10b468c9425b944a165c614 P dda8b5ea355c47b6ab2884509a4c085e: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:15.594571 28729 tablet_bootstrap.cc:492] T 932d03c8a10b468c9425b944a165c614 P dda8b5ea355c47b6ab2884509a4c085e: No bootstrap required, opened a new log
I20260929 10:59:15.594620 28729 ts_tablet_manager.cc:1397] T 932d03c8a10b468c9425b944a165c614 P dda8b5ea355c47b6ab2884509a4c085e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:59:15.595014 28726 raft_consensus.cc:348] T 932d03c8a10b468c9425b944a165c614 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } }
I20260929 10:59:15.595155 28726 raft_consensus.cc:374] T 932d03c8a10b468c9425b944a165c614 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:15.595199 28726 raft_consensus.cc:729] T 932d03c8a10b468c9425b944a165c614 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dbb8f2d2c1d4e9e8c467c2301e71ba6, State: Initialized, Role: FOLLOWER
I20260929 10:59:15.595314 28726 consensus_queue.cc:260] T 932d03c8a10b468c9425b944a165c614 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } }
I20260929 10:59:15.595556 28726 ts_tablet_manager.cc:1428] T 932d03c8a10b468c9425b944a165c614 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 10:59:15.595661 28422 heartbeater.cc:503] Master 127.26.120.254:46489 was elected leader, sending a full tablet report...
I20260929 10:59:15.596297 28729 raft_consensus.cc:348] T 932d03c8a10b468c9425b944a165c614 P dda8b5ea355c47b6ab2884509a4c085e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } }
I20260929 10:59:15.596453 28729 raft_consensus.cc:374] T 932d03c8a10b468c9425b944a165c614 P dda8b5ea355c47b6ab2884509a4c085e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:15.596503 28729 raft_consensus.cc:729] T 932d03c8a10b468c9425b944a165c614 P dda8b5ea355c47b6ab2884509a4c085e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dda8b5ea355c47b6ab2884509a4c085e, State: Initialized, Role: FOLLOWER
I20260929 10:59:15.596643 28729 consensus_queue.cc:260] T 932d03c8a10b468c9425b944a165c614 P dda8b5ea355c47b6ab2884509a4c085e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } }
I20260929 10:59:15.596911 28729 ts_tablet_manager.cc:1428] T 932d03c8a10b468c9425b944a165c614 P dda8b5ea355c47b6ab2884509a4c085e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:59:15.596931 28702 heartbeater.cc:503] Master 127.26.120.254:46489 was elected leader, sending a full tablet report...
W20260929 10:59:15.630642 28563 tablet.cc:2369] T 932d03c8a10b468c9425b944a165c614 P 571b683415fe42e194f078c902ed0f10: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:59:15.718664 28423 tablet.cc:2369] T 932d03c8a10b468c9425b944a165c614 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:59:15.791550 28703 tablet.cc:2369] T 932d03c8a10b468c9425b944a165c614 P dda8b5ea355c47b6ab2884509a4c085e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:59:15.872540 28732 raft_consensus.cc:482] T 932d03c8a10b468c9425b944a165c614 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:59:15.872716 28732 raft_consensus.cc:504] T 932d03c8a10b468c9425b944a165c614 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } }
I20260929 10:59:15.873124 28732 leader_election.cc:288] T 932d03c8a10b468c9425b944a165c614 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 571b683415fe42e194f078c902ed0f10 (127.26.120.194:37997), dda8b5ea355c47b6ab2884509a4c085e (127.26.120.195:43079)
I20260929 10:59:15.876194 28724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:15.873510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:52699 (local address 127.26.120.195:43079)
0929 10:59:15.873583 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:15.873585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:15.873595 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:59:15.873665 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:15.873666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:15.873673 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:15.873745 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:15.873932 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.874473 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.875431 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.875816 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.875882 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:15.875919 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:15.876071 (+ 152us) server_negotiation.cc:299] Negotiation successful
0929 10:59:15.876115 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 10:59:15.876223 28722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:15.873389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:54023 (local address 127.26.120.194:37997)
0929 10:59:15.873445 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:15.873447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:15.873495 (+ 48us) server_negotiation.cc:407] Connection header received
0929 10:59:15.873638 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:15.873640 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:15.873650 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:15.873711 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:15.873885 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.874417 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.875473 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.875830 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.875926 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:15.875933 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:15.876118 (+ 185us) server_negotiation.cc:299] Negotiation successful
0929 10:59:15.876162 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260929 10:59:15.876231 28735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:15.873401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:43079 (local address 127.26.120.193:52699)
0929 10:59:15.873499 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:15.873510 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:15.873601 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:15.873767 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:15.873768 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:15.873770 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:15.873837 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:15.873838 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.874566 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.874567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:15.875336 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:15.875338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.875915 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.875916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:15.875998 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:15.876001 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:59:15.876045 (+ 44us) client_negotiation.cc:240] Negotiation successful
0929 10:59:15.876102 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"mutex_wait_us":4,"thread_start_us":54,"threads_started":1}
I20260929 10:59:15.876240 28734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:15.873303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:37997 (local address 127.26.120.193:54023)
0929 10:59:15.873418 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:15.873432 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:15.873551 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:15.873729 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:15.873730 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:15.873732 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:15.873809 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:15.873816 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.874496 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.874497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:15.875348 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:15.875350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.875948 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.875949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:15.876016 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:15.876019 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:59:15.876093 (+ 74us) client_negotiation.cc:240] Negotiation successful
0929 10:59:15.876135 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"mutex_wait_us":30,"thread_start_us":64,"threads_started":1}
I20260929 10:59:15.876596 28656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "932d03c8a10b468c9425b944a165c614" candidate_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dda8b5ea355c47b6ab2884509a4c085e" is_pre_election: true
I20260929 10:59:15.876776 28656 raft_consensus.cc:2454] T 932d03c8a10b468c9425b944a165c614 P dda8b5ea355c47b6ab2884509a4c085e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dbb8f2d2c1d4e9e8c467c2301e71ba6 in term 0.
I20260929 10:59:15.876765 28516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "932d03c8a10b468c9425b944a165c614" candidate_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "571b683415fe42e194f078c902ed0f10" is_pre_election: true
I20260929 10:59:15.876914 28516 raft_consensus.cc:2454] T 932d03c8a10b468c9425b944a165c614 P 571b683415fe42e194f078c902ed0f10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dbb8f2d2c1d4e9e8c467c2301e71ba6 in term 0.
I20260929 10:59:15.877023 28310 leader_election.cc:302] T 932d03c8a10b468c9425b944a165c614 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dbb8f2d2c1d4e9e8c467c2301e71ba6, dda8b5ea355c47b6ab2884509a4c085e; no voters:
I20260929 10:59:15.877154 28732 raft_consensus.cc:2781] T 932d03c8a10b468c9425b944a165c614 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:59:15.877223 28732 raft_consensus.cc:482] T 932d03c8a10b468c9425b944a165c614 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:59:15.877256 28732 raft_consensus.cc:3037] T 932d03c8a10b468c9425b944a165c614 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:15.877959 28732 raft_consensus.cc:504] T 932d03c8a10b468c9425b944a165c614 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } }
I20260929 10:59:15.878152 28732 leader_election.cc:288] T 932d03c8a10b468c9425b944a165c614 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [CANDIDATE]: Term 1 election: Requested vote from peers 571b683415fe42e194f078c902ed0f10 (127.26.120.194:37997), dda8b5ea355c47b6ab2884509a4c085e (127.26.120.195:43079)
I20260929 10:59:15.878298 28516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "932d03c8a10b468c9425b944a165c614" candidate_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "571b683415fe42e194f078c902ed0f10"
I20260929 10:59:15.878312 28656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "932d03c8a10b468c9425b944a165c614" candidate_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dda8b5ea355c47b6ab2884509a4c085e"
I20260929 10:59:15.878372 28516 raft_consensus.cc:3037] T 932d03c8a10b468c9425b944a165c614 P 571b683415fe42e194f078c902ed0f10 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:15.878412 28656 raft_consensus.cc:3037] T 932d03c8a10b468c9425b944a165c614 P dda8b5ea355c47b6ab2884509a4c085e [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:15.879101 28516 raft_consensus.cc:2454] T 932d03c8a10b468c9425b944a165c614 P 571b683415fe42e194f078c902ed0f10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dbb8f2d2c1d4e9e8c467c2301e71ba6 in term 1.
I20260929 10:59:15.879128 28656 raft_consensus.cc:2454] T 932d03c8a10b468c9425b944a165c614 P dda8b5ea355c47b6ab2884509a4c085e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dbb8f2d2c1d4e9e8c467c2301e71ba6 in term 1.
I20260929 10:59:15.879292 28310 leader_election.cc:302] T 932d03c8a10b468c9425b944a165c614 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dbb8f2d2c1d4e9e8c467c2301e71ba6, dda8b5ea355c47b6ab2884509a4c085e; no voters:
I20260929 10:59:15.879441 28732 raft_consensus.cc:2781] T 932d03c8a10b468c9425b944a165c614 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:59:15.879591 28732 raft_consensus.cc:686] T 932d03c8a10b468c9425b944a165c614 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [term 1 LEADER]: Becoming Leader. State: Replica: 4dbb8f2d2c1d4e9e8c467c2301e71ba6, State: Running, Role: LEADER
I20260929 10:59:15.879711 28732 consensus_queue.cc:237] T 932d03c8a10b468c9425b944a165c614 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } }
I20260929 10:59:15.880550 28219 catalog_manager.cc:5495] T 932d03c8a10b468c9425b944a165c614 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dbb8f2d2c1d4e9e8c467c2301e71ba6 (127.26.120.193). New cstate: current_term: 1 leader_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } health_report { overall_health: UNKNOWN } } }
I20260929 10:59:15.938882 28721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:15.936253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33576 (local address 127.26.120.193:35425)
0929 10:59:15.936337 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:15.936339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:15.936381 (+ 42us) server_negotiation.cc:407] Connection header received
0929 10:59:15.936415 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:15.936416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:15.936433 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:15.936518 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:59:15.936681 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.937156 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.937970 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:15.938221 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:15.938305 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:15.938521 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:59:15.938627 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:59:15.938718 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:15.938770 (+ 52us) server_negotiation.cc:299] Negotiation successful
0929 10:59:15.938795 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 10:59:15.940871 28656 raft_consensus.cc:1261] T 932d03c8a10b468c9425b944a165c614 P dda8b5ea355c47b6ab2884509a4c085e [term 1 FOLLOWER]: Refusing update from remote peer 4dbb8f2d2c1d4e9e8c467c2301e71ba6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:59:15.941232 28736 consensus_queue.cc:1035] T 932d03c8a10b468c9425b944a165c614 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:59:15.943635 28516 raft_consensus.cc:1261] T 932d03c8a10b468c9425b944a165c614 P 571b683415fe42e194f078c902ed0f10 [term 1 FOLLOWER]: Refusing update from remote peer 4dbb8f2d2c1d4e9e8c467c2301e71ba6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:59:15.944068 28736 consensus_queue.cc:1035] T 932d03c8a10b468c9425b944a165c614 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:59:15.944188 28739 mvcc.cc:205] Tried to move back new op lower bound from 7334623461130760192 to 7334623460883759104. Current Snapshot: MvccSnapshot[applied={T|T < 7334623461130760192 or (T in {7334623461130760192})}]
I20260929 10:59:15.944291 28740 mvcc.cc:205] Tried to move back new op lower bound from 7334623461130760192 to 7334623460883759104. Current Snapshot: MvccSnapshot[applied={T|T < 7334623461130760192 or (T in {7334623461130760192})}]
I20260929 10:59:15.946475 28744 mvcc.cc:205] Tried to move back new op lower bound from 7334623461130760192 to 7334623460883759104. Current Snapshot: MvccSnapshot[applied={T|T < 7334623461130760192}]
I20260929 10:59:15.947554 28218 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:59:15.947650 28218 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:59:15.948518 28218 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5c435fdf336a4883b3edbc6b75ee9e73: Sending DeleteTablet for 3 replicas of tablet 932d03c8a10b468c9425b944a165c614
I20260929 10:59:15.948990 28496 tablet_service.cc:1514] Processing DeleteTablet for tablet 932d03c8a10b468c9425b944a165c614 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:15 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:59460
I20260929 10:59:15.948978 28356 tablet_service.cc:1514] Processing DeleteTablet for tablet 932d03c8a10b468c9425b944a165c614 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:15 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:33560
I20260929 10:59:15.949189 28622 tablet_service.cc:1514] Processing DeleteTablet for tablet 932d03c8a10b468c9425b944a165c614 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:15 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:36678
I20260929 10:59:15.949380 28748 tablet_replica.cc:330] T 932d03c8a10b468c9425b944a165c614 P 571b683415fe42e194f078c902ed0f10: stopping tablet replica
I20260929 10:59:15.949410 28749 tablet_replica.cc:330] T 932d03c8a10b468c9425b944a165c614 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6: stopping tablet replica
I20260929 10:59:15.949499 28749 raft_consensus.cc:2229] T 932d03c8a10b468c9425b944a165c614 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:59:15.949476 28218 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36830:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:59:15.949630 28218 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:59:15.949631 28749 raft_consensus.cc:2258] T 932d03c8a10b468c9425b944a165c614 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:15.950124 28750 tablet_replica.cc:330] T 932d03c8a10b468c9425b944a165c614 P dda8b5ea355c47b6ab2884509a4c085e: stopping tablet replica
I20260929 10:59:15.950234 28750 raft_consensus.cc:2229] T 932d03c8a10b468c9425b944a165c614 P dda8b5ea355c47b6ab2884509a4c085e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:59:15.950352 28750 raft_consensus.cc:2258] T 932d03c8a10b468c9425b944a165c614 P dda8b5ea355c47b6ab2884509a4c085e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:15.950688 28749 ts_tablet_manager.cc:1905] T 932d03c8a10b468c9425b944a165c614 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:15.950733 28748 raft_consensus.cc:2229] T 932d03c8a10b468c9425b944a165c614 P 571b683415fe42e194f078c902ed0f10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:59:15.950831 28748 raft_consensus.cc:2258] T 932d03c8a10b468c9425b944a165c614 P 571b683415fe42e194f078c902ed0f10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:15.950976 28496 tablet_service.cc:1467] Processing CreateTablet for tablet 210caeee111147e9904ce39a60faab42 (DEFAULT_TABLE table=test-table [id=5a8482d33c35443d8a719b6a1e0b13a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:15.951001 28356 tablet_service.cc:1467] Processing CreateTablet for tablet 210caeee111147e9904ce39a60faab42 (DEFAULT_TABLE table=test-table [id=5a8482d33c35443d8a719b6a1e0b13a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:15.951066 28622 tablet_service.cc:1467] Processing CreateTablet for tablet 210caeee111147e9904ce39a60faab42 (DEFAULT_TABLE table=test-table [id=5a8482d33c35443d8a719b6a1e0b13a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:15.951112 28496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 210caeee111147e9904ce39a60faab42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:15.951126 28356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 210caeee111147e9904ce39a60faab42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:15.951156 28622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 210caeee111147e9904ce39a60faab42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:15.952468 28726 tablet_bootstrap.cc:492] T 210caeee111147e9904ce39a60faab42 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6: Bootstrap starting.
I20260929 10:59:15.952517 28725 tablet_bootstrap.cc:492] T 210caeee111147e9904ce39a60faab42 P 571b683415fe42e194f078c902ed0f10: Bootstrap starting.
I20260929 10:59:15.952903 28726 tablet_bootstrap.cc:654] T 210caeee111147e9904ce39a60faab42 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:15.952922 28725 tablet_bootstrap.cc:654] T 210caeee111147e9904ce39a60faab42 P 571b683415fe42e194f078c902ed0f10: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:15.953254 28729 tablet_bootstrap.cc:492] T 210caeee111147e9904ce39a60faab42 P dda8b5ea355c47b6ab2884509a4c085e: Bootstrap starting.
I20260929 10:59:15.953787 28725 tablet_bootstrap.cc:492] T 210caeee111147e9904ce39a60faab42 P 571b683415fe42e194f078c902ed0f10: No bootstrap required, opened a new log
I20260929 10:59:15.953840 28725 ts_tablet_manager.cc:1397] T 210caeee111147e9904ce39a60faab42 P 571b683415fe42e194f078c902ed0f10: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 10:59:15.954020 28725 raft_consensus.cc:348] T 210caeee111147e9904ce39a60faab42 P 571b683415fe42e194f078c902ed0f10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } }
I20260929 10:59:15.954116 28725 raft_consensus.cc:374] T 210caeee111147e9904ce39a60faab42 P 571b683415fe42e194f078c902ed0f10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:15.954154 28725 raft_consensus.cc:729] T 210caeee111147e9904ce39a60faab42 P 571b683415fe42e194f078c902ed0f10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 571b683415fe42e194f078c902ed0f10, State: Initialized, Role: FOLLOWER
I20260929 10:59:15.954192 28725 consensus_queue.cc:260] T 210caeee111147e9904ce39a60faab42 P 571b683415fe42e194f078c902ed0f10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } }
I20260929 10:59:15.954212 28748 ts_tablet_manager.cc:1905] T 932d03c8a10b468c9425b944a165c614 P 571b683415fe42e194f078c902ed0f10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:15.954308 28725 ts_tablet_manager.cc:1428] T 210caeee111147e9904ce39a60faab42 P 571b683415fe42e194f078c902ed0f10: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:15.954435 28726 tablet_bootstrap.cc:492] T 210caeee111147e9904ce39a60faab42 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6: No bootstrap required, opened a new log
I20260929 10:59:15.954483 28726 ts_tablet_manager.cc:1397] T 210caeee111147e9904ce39a60faab42 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:59:15.954650 28726 raft_consensus.cc:348] T 210caeee111147e9904ce39a60faab42 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } }
I20260929 10:59:15.954725 28726 raft_consensus.cc:374] T 210caeee111147e9904ce39a60faab42 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:15.954746 28726 raft_consensus.cc:729] T 210caeee111147e9904ce39a60faab42 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dbb8f2d2c1d4e9e8c467c2301e71ba6, State: Initialized, Role: FOLLOWER
I20260929 10:59:15.954793 28726 consensus_queue.cc:260] T 210caeee111147e9904ce39a60faab42 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } }
I20260929 10:59:15.954886 28726 ts_tablet_manager.cc:1428] T 210caeee111147e9904ce39a60faab42 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260929 10:59:15.955638 28729 tablet_bootstrap.cc:654] T 210caeee111147e9904ce39a60faab42 P dda8b5ea355c47b6ab2884509a4c085e: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:15.956295 28729 tablet_bootstrap.cc:492] T 210caeee111147e9904ce39a60faab42 P dda8b5ea355c47b6ab2884509a4c085e: No bootstrap required, opened a new log
I20260929 10:59:15.956316 28748 ts_tablet_manager.cc:1918] T 932d03c8a10b468c9425b944a165c614 P 571b683415fe42e194f078c902ed0f10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:59:15.956351 28729 ts_tablet_manager.cc:1397] T 210caeee111147e9904ce39a60faab42 P dda8b5ea355c47b6ab2884509a4c085e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260929 10:59:15.956375 28748 log.cc:1196] T 932d03c8a10b468c9425b944a165c614 P 571b683415fe42e194f078c902ed0f10: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-1/wal/wals/932d03c8a10b468c9425b944a165c614
I20260929 10:59:15.956507 28729 raft_consensus.cc:348] T 210caeee111147e9904ce39a60faab42 P dda8b5ea355c47b6ab2884509a4c085e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } }
I20260929 10:59:15.956519 28750 ts_tablet_manager.cc:1905] T 932d03c8a10b468c9425b944a165c614 P dda8b5ea355c47b6ab2884509a4c085e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:15.956599 28729 raft_consensus.cc:374] T 210caeee111147e9904ce39a60faab42 P dda8b5ea355c47b6ab2884509a4c085e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:15.956629 28729 raft_consensus.cc:729] T 210caeee111147e9904ce39a60faab42 P dda8b5ea355c47b6ab2884509a4c085e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dda8b5ea355c47b6ab2884509a4c085e, State: Initialized, Role: FOLLOWER
I20260929 10:59:15.956655 28748 ts_tablet_manager.cc:1939] T 932d03c8a10b468c9425b944a165c614 P 571b683415fe42e194f078c902ed0f10: Deleting consensus metadata
I20260929 10:59:15.956682 28729 consensus_queue.cc:260] T 210caeee111147e9904ce39a60faab42 P dda8b5ea355c47b6ab2884509a4c085e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } }
I20260929 10:59:15.956799 28729 ts_tablet_manager.cc:1428] T 210caeee111147e9904ce39a60faab42 P dda8b5ea355c47b6ab2884509a4c085e: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260929 10:59:15.958297 28205 catalog_manager.cc:4841] TS 571b683415fe42e194f078c902ed0f10 (127.26.120.194:37997): tablet 932d03c8a10b468c9425b944a165c614 (table test-table [id=1f1fca2f86104b738398135d1abcf82f]) successfully deleted
I20260929 10:59:15.958324 28750 ts_tablet_manager.cc:1918] T 932d03c8a10b468c9425b944a165c614 P dda8b5ea355c47b6ab2884509a4c085e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:59:15.958395 28750 log.cc:1196] T 932d03c8a10b468c9425b944a165c614 P dda8b5ea355c47b6ab2884509a4c085e: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-2/wal/wals/932d03c8a10b468c9425b944a165c614
I20260929 10:59:15.958426 28749 ts_tablet_manager.cc:1918] T 932d03c8a10b468c9425b944a165c614 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:59:15.958501 28749 log.cc:1196] T 932d03c8a10b468c9425b944a165c614 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-0/wal/wals/932d03c8a10b468c9425b944a165c614
I20260929 10:59:15.958690 28750 ts_tablet_manager.cc:1939] T 932d03c8a10b468c9425b944a165c614 P dda8b5ea355c47b6ab2884509a4c085e: Deleting consensus metadata
I20260929 10:59:15.958778 28749 ts_tablet_manager.cc:1939] T 932d03c8a10b468c9425b944a165c614 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6: Deleting consensus metadata
I20260929 10:59:15.959123 28206 catalog_manager.cc:4841] TS dda8b5ea355c47b6ab2884509a4c085e (127.26.120.195:43079): tablet 932d03c8a10b468c9425b944a165c614 (table test-table [id=1f1fca2f86104b738398135d1abcf82f]) successfully deleted
I20260929 10:59:15.959215 28206 catalog_manager.cc:4841] TS 4dbb8f2d2c1d4e9e8c467c2301e71ba6 (127.26.120.193:35425): tablet 932d03c8a10b468c9425b944a165c614 (table test-table [id=1f1fca2f86104b738398135d1abcf82f]) successfully deleted
I20260929 10:59:16.200666 28733 raft_consensus.cc:482] T 210caeee111147e9904ce39a60faab42 P dda8b5ea355c47b6ab2884509a4c085e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:59:16.200804 28733 raft_consensus.cc:504] T 210caeee111147e9904ce39a60faab42 P dda8b5ea355c47b6ab2884509a4c085e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } }
I20260929 10:59:16.201190 28733 leader_election.cc:288] T 210caeee111147e9904ce39a60faab42 P dda8b5ea355c47b6ab2884509a4c085e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dbb8f2d2c1d4e9e8c467c2301e71ba6 (127.26.120.193:35425), 571b683415fe42e194f078c902ed0f10 (127.26.120.194:37997)
I20260929 10:59:16.204276 28722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:16.201456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:37665 (local address 127.26.120.194:37997)
0929 10:59:16.201501 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:16.201503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:16.201610 (+ 107us) server_negotiation.cc:407] Connection header received
0929 10:59:16.201714 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:16.201715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:16.201731 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:16.201797 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:16.201968 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:16.202492 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:16.203537 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:16.203897 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:16.203984 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:16.203994 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:16.204160 (+ 166us) server_negotiation.cc:299] Negotiation successful
0929 10:59:16.204194 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 10:59:16.204288 28721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:16.201434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:32805 (local address 127.26.120.193:35425)
0929 10:59:16.201484 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:16.201487 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:16.201619 (+ 132us) server_negotiation.cc:407] Connection header received
0929 10:59:16.201714 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:16.201716 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:16.201734 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:16.201791 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:16.201965 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:16.202452 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:16.203563 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:16.203928 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:16.204016 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:16.204032 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:16.204175 (+ 143us) server_negotiation.cc:299] Negotiation successful
0929 10:59:16.204214 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 10:59:16.204322 28754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:16.201379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:37997 (local address 127.26.120.195:37665)
0929 10:59:16.201506 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:16.201516 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:16.201607 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:16.201805 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:16.201806 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:16.201809 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:16.201898 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:16.201900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:16.202576 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:16.202577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:16.203444 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:16.203446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:16.203989 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:16.203990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:16.204072 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:16.204076 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:59:16.204151 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 10:59:16.204194 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":4,"thread_start_us":65,"threads_started":1}
I20260929 10:59:16.204324 28755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:16.201379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:35425 (local address 127.26.120.195:32805)
0929 10:59:16.201543 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:16.201552 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:16.201645 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:16.201812 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:16.201812 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:16.201815 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:16.201893 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:16.201895 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:16.202594 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:16.202594 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:16.203445 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:16.203447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:16.204016 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:16.204017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:16.204097 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:16.204100 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:59:16.204168 (+ 68us) client_negotiation.cc:240] Negotiation successful
0929 10:59:16.204212 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":113,"thread_start_us":54,"threads_started":1}
I20260929 10:59:16.204604 28515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "210caeee111147e9904ce39a60faab42" candidate_uuid: "dda8b5ea355c47b6ab2884509a4c085e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "571b683415fe42e194f078c902ed0f10" is_pre_election: true
I20260929 10:59:16.204694 28515 raft_consensus.cc:2454] T 210caeee111147e9904ce39a60faab42 P 571b683415fe42e194f078c902ed0f10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dda8b5ea355c47b6ab2884509a4c085e in term 0.
I20260929 10:59:16.204787 28376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "210caeee111147e9904ce39a60faab42" candidate_uuid: "dda8b5ea355c47b6ab2884509a4c085e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" is_pre_election: true
I20260929 10:59:16.204847 28589 leader_election.cc:302] T 210caeee111147e9904ce39a60faab42 P dda8b5ea355c47b6ab2884509a4c085e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 571b683415fe42e194f078c902ed0f10, dda8b5ea355c47b6ab2884509a4c085e; no voters:
I20260929 10:59:16.204931 28376 raft_consensus.cc:2454] T 210caeee111147e9904ce39a60faab42 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dda8b5ea355c47b6ab2884509a4c085e in term 0.
I20260929 10:59:16.204957 28733 raft_consensus.cc:2781] T 210caeee111147e9904ce39a60faab42 P dda8b5ea355c47b6ab2884509a4c085e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:59:16.204996 28733 raft_consensus.cc:482] T 210caeee111147e9904ce39a60faab42 P dda8b5ea355c47b6ab2884509a4c085e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:59:16.205056 28733 raft_consensus.cc:3037] T 210caeee111147e9904ce39a60faab42 P dda8b5ea355c47b6ab2884509a4c085e [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:16.206073 28733 raft_consensus.cc:504] T 210caeee111147e9904ce39a60faab42 P dda8b5ea355c47b6ab2884509a4c085e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } }
I20260929 10:59:16.206218 28733 leader_election.cc:288] T 210caeee111147e9904ce39a60faab42 P dda8b5ea355c47b6ab2884509a4c085e [CANDIDATE]: Term 1 election: Requested vote from peers 4dbb8f2d2c1d4e9e8c467c2301e71ba6 (127.26.120.193:35425), 571b683415fe42e194f078c902ed0f10 (127.26.120.194:37997)
I20260929 10:59:16.206453 28515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "210caeee111147e9904ce39a60faab42" candidate_uuid: "dda8b5ea355c47b6ab2884509a4c085e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "571b683415fe42e194f078c902ed0f10"
I20260929 10:59:16.206453 28376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "210caeee111147e9904ce39a60faab42" candidate_uuid: "dda8b5ea355c47b6ab2884509a4c085e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6"
I20260929 10:59:16.206540 28376 raft_consensus.cc:3037] T 210caeee111147e9904ce39a60faab42 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:16.206538 28515 raft_consensus.cc:3037] T 210caeee111147e9904ce39a60faab42 P 571b683415fe42e194f078c902ed0f10 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:16.207331 28515 raft_consensus.cc:2454] T 210caeee111147e9904ce39a60faab42 P 571b683415fe42e194f078c902ed0f10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dda8b5ea355c47b6ab2884509a4c085e in term 1.
I20260929 10:59:16.207350 28376 raft_consensus.cc:2454] T 210caeee111147e9904ce39a60faab42 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dda8b5ea355c47b6ab2884509a4c085e in term 1.
I20260929 10:59:16.207558 28590 leader_election.cc:302] T 210caeee111147e9904ce39a60faab42 P dda8b5ea355c47b6ab2884509a4c085e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dbb8f2d2c1d4e9e8c467c2301e71ba6, dda8b5ea355c47b6ab2884509a4c085e; no voters:
I20260929 10:59:16.207650 28733 raft_consensus.cc:2781] T 210caeee111147e9904ce39a60faab42 P dda8b5ea355c47b6ab2884509a4c085e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:59:16.207759 28733 raft_consensus.cc:686] T 210caeee111147e9904ce39a60faab42 P dda8b5ea355c47b6ab2884509a4c085e [term 1 LEADER]: Becoming Leader. State: Replica: dda8b5ea355c47b6ab2884509a4c085e, State: Running, Role: LEADER
I20260929 10:59:16.207830 28733 consensus_queue.cc:237] T 210caeee111147e9904ce39a60faab42 P dda8b5ea355c47b6ab2884509a4c085e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } }
I20260929 10:59:16.208559 28218 catalog_manager.cc:5495] T 210caeee111147e9904ce39a60faab42 P dda8b5ea355c47b6ab2884509a4c085e reported cstate change: term changed from 0 to 1, leader changed from <none> to dda8b5ea355c47b6ab2884509a4c085e (127.26.120.195). New cstate: current_term: 1 leader_uuid: "dda8b5ea355c47b6ab2884509a4c085e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } health_report { overall_health: HEALTHY } } }
Sep 29 10:59:16 dist-test-slave-2lzq krb5kdc[28174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679555, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:16.258008 28764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:16.250735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:46489 (local address 127.0.0.1:36844)
0929 10:59:16.250951 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:16.250961 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:16.251018 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:16.251171 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:16.251172 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:16.251476 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:16.251541 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:16.251543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:16.252203 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:16.252203 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:16.252740 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:16.252742 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:16.253205 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:16.253206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:16.253249 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:16.253825 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:16.253829 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:16.255633 (+ 1804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:16.257069 (+ 1436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:16.257076 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:16.257079 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:16.257271 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:16.257471 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:16.257472 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:16.257472 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:16.257541 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:16.257747 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:16.257753 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:16.257793 (+ 40us) client_negotiation.cc:769] Sending connection context
0929 10:59:16.257845 (+ 52us) client_negotiation.cc:240] Negotiation successful
0929 10:59:16.257933 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":141,"threads_started":1}
I20260929 10:59:16.258174 28763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:16.250802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36844 (local address 127.26.120.254:46489)
0929 10:59:16.250985 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:16.250987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:16.251001 (+ 14us) server_negotiation.cc:407] Connection header received
0929 10:59:16.251080 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:16.251080 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:16.251090 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:16.251156 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:16.251613 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:16.252131 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:16.252854 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:16.253138 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:16.253204 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:16.255729 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:16.255731 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:16.255732 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:16.255740 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:16.256996 (+ 1256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:16.257346 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:16.257347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:16.257347 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:16.257400 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:16.257594 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:16.257595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:16.257595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:16.257694 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:16.257743 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:16.257876 (+ 133us) server_negotiation.cc:299] Negotiation successful
0929 10:59:16.257983 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":161,"thread_start_us":78,"threads_started":1}
I20260929 10:59:16.260107 28218 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:36844:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 10:59:16.260314 28218 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:59:16.261853 28356 tablet_service.cc:1467] Processing CreateTablet for tablet bb1f0b22a2014815a03b75566460d8c1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9668b348637243918d501c7cf7c1f592]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 10:59:16.261909 28496 tablet_service.cc:1467] Processing CreateTablet for tablet bb1f0b22a2014815a03b75566460d8c1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9668b348637243918d501c7cf7c1f592]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 10:59:16.262019 28356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb1f0b22a2014815a03b75566460d8c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:16.262033 28496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb1f0b22a2014815a03b75566460d8c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:16.261999 28622 tablet_service.cc:1467] Processing CreateTablet for tablet bb1f0b22a2014815a03b75566460d8c1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9668b348637243918d501c7cf7c1f592]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 10:59:16.262174 28622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb1f0b22a2014815a03b75566460d8c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:16.263209 28729 tablet_bootstrap.cc:492] T bb1f0b22a2014815a03b75566460d8c1 P dda8b5ea355c47b6ab2884509a4c085e: Bootstrap starting.
I20260929 10:59:16.263535 28726 tablet_bootstrap.cc:492] T bb1f0b22a2014815a03b75566460d8c1 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6: Bootstrap starting.
I20260929 10:59:16.263612 28729 tablet_bootstrap.cc:654] T bb1f0b22a2014815a03b75566460d8c1 P dda8b5ea355c47b6ab2884509a4c085e: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:16.263852 28726 tablet_bootstrap.cc:654] T bb1f0b22a2014815a03b75566460d8c1 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:16.264119 28725 tablet_bootstrap.cc:492] T bb1f0b22a2014815a03b75566460d8c1 P 571b683415fe42e194f078c902ed0f10: Bootstrap starting.
I20260929 10:59:16.264315 28729 tablet_bootstrap.cc:492] T bb1f0b22a2014815a03b75566460d8c1 P dda8b5ea355c47b6ab2884509a4c085e: No bootstrap required, opened a new log
I20260929 10:59:16.264372 28729 ts_tablet_manager.cc:1397] T bb1f0b22a2014815a03b75566460d8c1 P dda8b5ea355c47b6ab2884509a4c085e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 10:59:16.264529 28725 tablet_bootstrap.cc:654] T bb1f0b22a2014815a03b75566460d8c1 P 571b683415fe42e194f078c902ed0f10: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:16.264535 28729 raft_consensus.cc:348] T bb1f0b22a2014815a03b75566460d8c1 P dda8b5ea355c47b6ab2884509a4c085e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } }
I20260929 10:59:16.264609 28729 raft_consensus.cc:374] T bb1f0b22a2014815a03b75566460d8c1 P dda8b5ea355c47b6ab2884509a4c085e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:16.264647 28729 raft_consensus.cc:729] T bb1f0b22a2014815a03b75566460d8c1 P dda8b5ea355c47b6ab2884509a4c085e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dda8b5ea355c47b6ab2884509a4c085e, State: Initialized, Role: FOLLOWER
I20260929 10:59:16.264703 28729 consensus_queue.cc:260] T bb1f0b22a2014815a03b75566460d8c1 P dda8b5ea355c47b6ab2884509a4c085e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } }
I20260929 10:59:16.264819 28729 ts_tablet_manager.cc:1428] T bb1f0b22a2014815a03b75566460d8c1 P dda8b5ea355c47b6ab2884509a4c085e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:16.264878 28733 tablet_replica.cc:439] T bb1f0b22a2014815a03b75566460d8c1 P dda8b5ea355c47b6ab2884509a4c085e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } } }
I20260929 10:59:16.264987 28733 tablet_replica.cc:442] T bb1f0b22a2014815a03b75566460d8c1 P dda8b5ea355c47b6ab2884509a4c085e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:59:16.265130 28726 tablet_bootstrap.cc:492] T bb1f0b22a2014815a03b75566460d8c1 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6: No bootstrap required, opened a new log
I20260929 10:59:16.265187 28726 ts_tablet_manager.cc:1397] T bb1f0b22a2014815a03b75566460d8c1 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:59:16.265309 28726 raft_consensus.cc:348] T bb1f0b22a2014815a03b75566460d8c1 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } }
I20260929 10:59:16.265388 28726 raft_consensus.cc:374] T bb1f0b22a2014815a03b75566460d8c1 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:16.265388 28725 tablet_bootstrap.cc:492] T bb1f0b22a2014815a03b75566460d8c1 P 571b683415fe42e194f078c902ed0f10: No bootstrap required, opened a new log
I20260929 10:59:16.265410 28726 raft_consensus.cc:729] T bb1f0b22a2014815a03b75566460d8c1 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dbb8f2d2c1d4e9e8c467c2301e71ba6, State: Initialized, Role: FOLLOWER
I20260929 10:59:16.265429 28725 ts_tablet_manager.cc:1397] T bb1f0b22a2014815a03b75566460d8c1 P 571b683415fe42e194f078c902ed0f10: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:59:16.265491 28726 consensus_queue.cc:260] T bb1f0b22a2014815a03b75566460d8c1 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } }
I20260929 10:59:16.265604 28726 ts_tablet_manager.cc:1428] T bb1f0b22a2014815a03b75566460d8c1 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:16.265580 28725 raft_consensus.cc:348] T bb1f0b22a2014815a03b75566460d8c1 P 571b683415fe42e194f078c902ed0f10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } }
I20260929 10:59:16.265645 28725 raft_consensus.cc:374] T bb1f0b22a2014815a03b75566460d8c1 P 571b683415fe42e194f078c902ed0f10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:16.265668 28725 raft_consensus.cc:729] T bb1f0b22a2014815a03b75566460d8c1 P 571b683415fe42e194f078c902ed0f10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 571b683415fe42e194f078c902ed0f10, State: Initialized, Role: FOLLOWER
I20260929 10:59:16.265627 28736 tablet_replica.cc:439] T bb1f0b22a2014815a03b75566460d8c1 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } } }
I20260929 10:59:16.265712 28725 consensus_queue.cc:260] T bb1f0b22a2014815a03b75566460d8c1 P 571b683415fe42e194f078c902ed0f10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } }
I20260929 10:59:16.265731 28736 tablet_replica.cc:442] T bb1f0b22a2014815a03b75566460d8c1 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:59:16.265812 28725 ts_tablet_manager.cc:1428] T bb1f0b22a2014815a03b75566460d8c1 P 571b683415fe42e194f078c902ed0f10: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:16.265873 28730 tablet_replica.cc:439] T bb1f0b22a2014815a03b75566460d8c1 P 571b683415fe42e194f078c902ed0f10: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } } }
I20260929 10:59:16.265959 28730 tablet_replica.cc:442] T bb1f0b22a2014815a03b75566460d8c1 P 571b683415fe42e194f078c902ed0f10: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:59:16.271070 28733 raft_consensus.cc:482] T bb1f0b22a2014815a03b75566460d8c1 P dda8b5ea355c47b6ab2884509a4c085e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:59:16.271126 28733 raft_consensus.cc:504] T bb1f0b22a2014815a03b75566460d8c1 P dda8b5ea355c47b6ab2884509a4c085e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } }
I20260929 10:59:16.271270 28733 leader_election.cc:288] T bb1f0b22a2014815a03b75566460d8c1 P dda8b5ea355c47b6ab2884509a4c085e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dbb8f2d2c1d4e9e8c467c2301e71ba6 (127.26.120.193:35425), 571b683415fe42e194f078c902ed0f10 (127.26.120.194:37997)
I20260929 10:59:16.271431 28376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb1f0b22a2014815a03b75566460d8c1" candidate_uuid: "dda8b5ea355c47b6ab2884509a4c085e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" is_pre_election: true
I20260929 10:59:16.271484 28515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb1f0b22a2014815a03b75566460d8c1" candidate_uuid: "dda8b5ea355c47b6ab2884509a4c085e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "571b683415fe42e194f078c902ed0f10" is_pre_election: true
I20260929 10:59:16.271535 28376 raft_consensus.cc:2454] T bb1f0b22a2014815a03b75566460d8c1 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dda8b5ea355c47b6ab2884509a4c085e in term 0.
I20260929 10:59:16.271548 28515 raft_consensus.cc:2454] T bb1f0b22a2014815a03b75566460d8c1 P 571b683415fe42e194f078c902ed0f10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dda8b5ea355c47b6ab2884509a4c085e in term 0.
I20260929 10:59:16.271668 28590 leader_election.cc:302] T bb1f0b22a2014815a03b75566460d8c1 P dda8b5ea355c47b6ab2884509a4c085e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dbb8f2d2c1d4e9e8c467c2301e71ba6, dda8b5ea355c47b6ab2884509a4c085e; no voters:
I20260929 10:59:16.271752 28733 raft_consensus.cc:2781] T bb1f0b22a2014815a03b75566460d8c1 P dda8b5ea355c47b6ab2884509a4c085e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:59:16.271798 28733 raft_consensus.cc:482] T bb1f0b22a2014815a03b75566460d8c1 P dda8b5ea355c47b6ab2884509a4c085e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:59:16.271828 28733 raft_consensus.cc:3037] T bb1f0b22a2014815a03b75566460d8c1 P dda8b5ea355c47b6ab2884509a4c085e [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:16.272563 28733 raft_consensus.cc:504] T bb1f0b22a2014815a03b75566460d8c1 P dda8b5ea355c47b6ab2884509a4c085e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } }
I20260929 10:59:16.272696 28733 leader_election.cc:288] T bb1f0b22a2014815a03b75566460d8c1 P dda8b5ea355c47b6ab2884509a4c085e [CANDIDATE]: Term 1 election: Requested vote from peers 4dbb8f2d2c1d4e9e8c467c2301e71ba6 (127.26.120.193:35425), 571b683415fe42e194f078c902ed0f10 (127.26.120.194:37997)
I20260929 10:59:16.272852 28376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb1f0b22a2014815a03b75566460d8c1" candidate_uuid: "dda8b5ea355c47b6ab2884509a4c085e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6"
I20260929 10:59:16.272877 28515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb1f0b22a2014815a03b75566460d8c1" candidate_uuid: "dda8b5ea355c47b6ab2884509a4c085e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "571b683415fe42e194f078c902ed0f10"
I20260929 10:59:16.272938 28515 raft_consensus.cc:3037] T bb1f0b22a2014815a03b75566460d8c1 P 571b683415fe42e194f078c902ed0f10 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:16.272930 28376 raft_consensus.cc:3037] T bb1f0b22a2014815a03b75566460d8c1 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:16.273458 28376 raft_consensus.cc:2454] T bb1f0b22a2014815a03b75566460d8c1 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dda8b5ea355c47b6ab2884509a4c085e in term 1.
I20260929 10:59:16.273514 28515 raft_consensus.cc:2454] T bb1f0b22a2014815a03b75566460d8c1 P 571b683415fe42e194f078c902ed0f10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dda8b5ea355c47b6ab2884509a4c085e in term 1.
I20260929 10:59:16.273623 28590 leader_election.cc:302] T bb1f0b22a2014815a03b75566460d8c1 P dda8b5ea355c47b6ab2884509a4c085e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dbb8f2d2c1d4e9e8c467c2301e71ba6, dda8b5ea355c47b6ab2884509a4c085e; no voters:
I20260929 10:59:16.273718 28733 raft_consensus.cc:2781] T bb1f0b22a2014815a03b75566460d8c1 P dda8b5ea355c47b6ab2884509a4c085e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:59:16.273777 28733 raft_consensus.cc:686] T bb1f0b22a2014815a03b75566460d8c1 P dda8b5ea355c47b6ab2884509a4c085e [term 1 LEADER]: Becoming Leader. State: Replica: dda8b5ea355c47b6ab2884509a4c085e, State: Running, Role: LEADER
I20260929 10:59:16.273824 28733 consensus_queue.cc:237] T bb1f0b22a2014815a03b75566460d8c1 P dda8b5ea355c47b6ab2884509a4c085e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } }
I20260929 10:59:16.274124 28733 tablet_replica.cc:439] T bb1f0b22a2014815a03b75566460d8c1 P dda8b5ea355c47b6ab2884509a4c085e: TxnStatusTablet state changed. Reason: New leader dda8b5ea355c47b6ab2884509a4c085e. Latest consensus state: current_term: 1 leader_uuid: "dda8b5ea355c47b6ab2884509a4c085e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } } }
I20260929 10:59:16.274211 28733 tablet_replica.cc:442] T bb1f0b22a2014815a03b75566460d8c1 P dda8b5ea355c47b6ab2884509a4c085e: This TxnStatusTablet replica's current role is: LEADER
I20260929 10:59:16.274380 28769 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 10:59:16.274471 28769 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 10:59:16.274395 28218 catalog_manager.cc:5495] T bb1f0b22a2014815a03b75566460d8c1 P dda8b5ea355c47b6ab2884509a4c085e reported cstate change: term changed from 0 to 1, leader changed from <none> to dda8b5ea355c47b6ab2884509a4c085e (127.26.120.195). New cstate: current_term: 1 leader_uuid: "dda8b5ea355c47b6ab2884509a4c085e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } health_report { overall_health: HEALTHY } } }
I20260929 10:59:16.280817 28764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:16.277734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:43079 (local address 127.0.0.1:36688)
0929 10:59:16.277813 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:16.277823 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:16.277857 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:16.278009 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:16.278010 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:16.278011 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 10:59:16.278106 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:16.278109 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:16.279047 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:16.279048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:16.279878 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:16.279882 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:16.280347 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:16.280348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:16.280388 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:16.280403 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 10:59:16.280680 (+ 277us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 10:59:16.280686 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 10:59:16.280722 (+ 36us) client_negotiation.cc:240] Negotiation successful
0929 10:59:16.280762 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260929 10:59:16.280833 28724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:16.277812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36688 (local address 127.26.120.195:43079)
0929 10:59:16.277856 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:16.277858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:16.277868 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:59:16.277907 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:16.277908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:16.277924 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:16.278002 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:59:16.278162 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:16.278928 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:16.279965 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:16.280268 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:16.280344 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:16.280492 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:59:16.280613 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:59:16.280690 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:16.280728 (+ 38us) server_negotiation.cc:299] Negotiation successful
0929 10:59:16.280753 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260929 10:59:16.281999 28515 raft_consensus.cc:1261] T bb1f0b22a2014815a03b75566460d8c1 P 571b683415fe42e194f078c902ed0f10 [term 1 FOLLOWER]: Refusing update from remote peer dda8b5ea355c47b6ab2884509a4c085e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:59:16.282207 28376 raft_consensus.cc:1261] T bb1f0b22a2014815a03b75566460d8c1 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [term 1 FOLLOWER]: Refusing update from remote peer dda8b5ea355c47b6ab2884509a4c085e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:59:16.282356 28733 consensus_queue.cc:1035] T bb1f0b22a2014815a03b75566460d8c1 P dda8b5ea355c47b6ab2884509a4c085e [LEADER]: Connected to new peer: Peer: permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:59:16.282586 28756 consensus_queue.cc:1035] T bb1f0b22a2014815a03b75566460d8c1 P dda8b5ea355c47b6ab2884509a4c085e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:59:16.283282 28736 tablet_replica.cc:439] T bb1f0b22a2014815a03b75566460d8c1 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6: TxnStatusTablet state changed. Reason: New leader dda8b5ea355c47b6ab2884509a4c085e. Latest consensus state: current_term: 1 leader_uuid: "dda8b5ea355c47b6ab2884509a4c085e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } } }
I20260929 10:59:16.283372 28736 tablet_replica.cc:442] T bb1f0b22a2014815a03b75566460d8c1 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:59:16.283540 28730 tablet_replica.cc:439] T bb1f0b22a2014815a03b75566460d8c1 P 571b683415fe42e194f078c902ed0f10: TxnStatusTablet state changed. Reason: New leader dda8b5ea355c47b6ab2884509a4c085e. Latest consensus state: current_term: 1 leader_uuid: "dda8b5ea355c47b6ab2884509a4c085e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } } }
I20260929 10:59:16.283655 28730 tablet_replica.cc:442] T bb1f0b22a2014815a03b75566460d8c1 P 571b683415fe42e194f078c902ed0f10: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:59:16.284106 28756 tablet_replica.cc:439] T bb1f0b22a2014815a03b75566460d8c1 P dda8b5ea355c47b6ab2884509a4c085e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dda8b5ea355c47b6ab2884509a4c085e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } } }
I20260929 10:59:16.284190 28756 tablet_replica.cc:442] T bb1f0b22a2014815a03b75566460d8c1 P dda8b5ea355c47b6ab2884509a4c085e: This TxnStatusTablet replica's current role is: LEADER
I20260929 10:59:16.284293 28756 tablet_replica.cc:439] T bb1f0b22a2014815a03b75566460d8c1 P dda8b5ea355c47b6ab2884509a4c085e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dda8b5ea355c47b6ab2884509a4c085e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } } }
I20260929 10:59:16.284346 28756 tablet_replica.cc:442] T bb1f0b22a2014815a03b75566460d8c1 P dda8b5ea355c47b6ab2884509a4c085e: This TxnStatusTablet replica's current role is: LEADER
I20260929 10:59:16.284751 28736 tablet_replica.cc:439] T bb1f0b22a2014815a03b75566460d8c1 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dda8b5ea355c47b6ab2884509a4c085e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } } }
I20260929 10:59:16.284847 28736 tablet_replica.cc:442] T bb1f0b22a2014815a03b75566460d8c1 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:59:16.284850 28730 tablet_replica.cc:439] T bb1f0b22a2014815a03b75566460d8c1 P 571b683415fe42e194f078c902ed0f10: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dda8b5ea355c47b6ab2884509a4c085e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 } } peers { permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 } } peers { permanent_uuid: "dda8b5ea355c47b6ab2884509a4c085e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43079 } } }
I20260929 10:59:16.284958 28730 tablet_replica.cc:442] T bb1f0b22a2014815a03b75566460d8c1 P 571b683415fe42e194f078c902ed0f10: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:59:16.288456 28724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:16.286129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36696 (local address 127.26.120.195:43079)
0929 10:59:16.286186 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:16.286188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:16.286196 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:59:16.286217 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:16.286218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:16.286229 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:16.286285 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:59:16.286366 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:16.287078 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:16.287768 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:16.288076 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:16.288147 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:16.288242 (+ 95us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:59:16.288274 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:59:16.288321 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:16.288365 (+ 44us) server_negotiation.cc:299] Negotiation successful
0929 10:59:16.288382 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 10:59:16.294013 28724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:16.291184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:46525 (local address 127.26.120.195:43079)
0929 10:59:16.291223 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:16.291225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:16.291363 (+ 138us) server_negotiation.cc:407] Connection header received
0929 10:59:16.291390 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:16.291391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:16.291400 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:16.291470 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:59:16.291651 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:16.292375 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:16.293250 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:16.293551 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:16.293619 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:16.293784 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:59:16.293816 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:59:16.293867 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:16.293911 (+ 44us) server_negotiation.cc:299] Negotiation successful
0929 10:59:16.293930 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":20}
I20260929 10:59:16.294134 28776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:16.291129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:43079 (local address 127.26.120.195:46525)
0929 10:59:16.291284 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:16.291295 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:16.291355 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:16.291472 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:16.291472 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:16.291473 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 10:59:16.291593 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:16.291595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:16.292441 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:16.292441 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:16.293172 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:16.293174 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:16.293623 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:16.293623 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:16.293697 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:16.293710 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 10:59:16.293873 (+ 163us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 10:59:16.293874 (+ 1us) client_negotiation.cc:769] Sending connection context
0929 10:59:16.293909 (+ 35us) client_negotiation.cc:240] Negotiation successful
0929 10:59:16.293943 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":94,"threads_started":1}
I20260929 10:59:16.296221 28515 raft_consensus.cc:1261] T 210caeee111147e9904ce39a60faab42 P 571b683415fe42e194f078c902ed0f10 [term 1 FOLLOWER]: Refusing update from remote peer dda8b5ea355c47b6ab2884509a4c085e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:59:16.296231 28376 raft_consensus.cc:1261] T 210caeee111147e9904ce39a60faab42 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [term 1 FOLLOWER]: Refusing update from remote peer dda8b5ea355c47b6ab2884509a4c085e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:59:16.296458 28756 consensus_queue.cc:1035] T 210caeee111147e9904ce39a60faab42 P dda8b5ea355c47b6ab2884509a4c085e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dbb8f2d2c1d4e9e8c467c2301e71ba6" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:59:16.296566 28756 consensus_queue.cc:1035] T 210caeee111147e9904ce39a60faab42 P dda8b5ea355c47b6ab2884509a4c085e [LEADER]: Connected to new peer: Peer: permanent_uuid: "571b683415fe42e194f078c902ed0f10" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:59:16.308240 28216 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:59:16.308357 28216 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:59:16.309079 28216 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5c435fdf336a4883b3edbc6b75ee9e73: Sending DeleteTablet for 3 replicas of tablet 210caeee111147e9904ce39a60faab42
I20260929 10:59:16.309334 28356 tablet_service.cc:1514] Processing DeleteTablet for tablet 210caeee111147e9904ce39a60faab42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:16 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:33560
I20260929 10:59:16.309453 28749 tablet_replica.cc:330] T 210caeee111147e9904ce39a60faab42 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6: stopping tablet replica
I20260929 10:59:16.309453 28496 tablet_service.cc:1514] Processing DeleteTablet for tablet 210caeee111147e9904ce39a60faab42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:16 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:59460
I20260929 10:59:16.309532 28749 raft_consensus.cc:2229] T 210caeee111147e9904ce39a60faab42 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:59:16.309561 28748 tablet_replica.cc:330] T 210caeee111147e9904ce39a60faab42 P 571b683415fe42e194f078c902ed0f10: stopping tablet replica
I20260929 10:59:16.309571 28749 raft_consensus.cc:2258] T 210caeee111147e9904ce39a60faab42 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:16.309600 28748 raft_consensus.cc:2229] T 210caeee111147e9904ce39a60faab42 P 571b683415fe42e194f078c902ed0f10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:59:16.309654 28748 raft_consensus.cc:2258] T 210caeee111147e9904ce39a60faab42 P 571b683415fe42e194f078c902ed0f10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:16.309962 28636 tablet_service.cc:1514] Processing DeleteTablet for tablet 210caeee111147e9904ce39a60faab42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:16 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:36678
I20260929 10:59:16.310141 28750 tablet_replica.cc:330] T 210caeee111147e9904ce39a60faab42 P dda8b5ea355c47b6ab2884509a4c085e: stopping tablet replica
I20260929 10:59:16.310200 28750 raft_consensus.cc:2229] T 210caeee111147e9904ce39a60faab42 P dda8b5ea355c47b6ab2884509a4c085e [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:59:16.310285 28750 raft_consensus.cc:2258] T 210caeee111147e9904ce39a60faab42 P dda8b5ea355c47b6ab2884509a4c085e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:16.310407 28748 ts_tablet_manager.cc:1905] T 210caeee111147e9904ce39a60faab42 P 571b683415fe42e194f078c902ed0f10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:16.310642 28750 ts_tablet_manager.cc:1905] T 210caeee111147e9904ce39a60faab42 P dda8b5ea355c47b6ab2884509a4c085e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:16.312180 28748 ts_tablet_manager.cc:1918] T 210caeee111147e9904ce39a60faab42 P 571b683415fe42e194f078c902ed0f10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:59:16.312247 28748 log.cc:1196] T 210caeee111147e9904ce39a60faab42 P 571b683415fe42e194f078c902ed0f10: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-1/wal/wals/210caeee111147e9904ce39a60faab42
I20260929 10:59:16.312342 28750 ts_tablet_manager.cc:1918] T 210caeee111147e9904ce39a60faab42 P dda8b5ea355c47b6ab2884509a4c085e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:59:16.312389 28750 log.cc:1196] T 210caeee111147e9904ce39a60faab42 P dda8b5ea355c47b6ab2884509a4c085e: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-2/wal/wals/210caeee111147e9904ce39a60faab42
I20260929 10:59:16.312469 28748 ts_tablet_manager.cc:1939] T 210caeee111147e9904ce39a60faab42 P 571b683415fe42e194f078c902ed0f10: Deleting consensus metadata
I20260929 10:59:16.312601 28750 ts_tablet_manager.cc:1939] T 210caeee111147e9904ce39a60faab42 P dda8b5ea355c47b6ab2884509a4c085e: Deleting consensus metadata
I20260929 10:59:16.312891 28205 catalog_manager.cc:4841] TS 571b683415fe42e194f078c902ed0f10 (127.26.120.194:37997): tablet 210caeee111147e9904ce39a60faab42 (table test-table [id=5a8482d33c35443d8a719b6a1e0b13a3]) successfully deleted
I20260929 10:59:16.312916 28206 catalog_manager.cc:4841] TS dda8b5ea355c47b6ab2884509a4c085e (127.26.120.195:43079): tablet 210caeee111147e9904ce39a60faab42 (table test-table [id=5a8482d33c35443d8a719b6a1e0b13a3]) successfully deleted
I20260929 10:59:16.312943 28749 ts_tablet_manager.cc:1905] T 210caeee111147e9904ce39a60faab42 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:16.314043 28749 ts_tablet_manager.cc:1918] T 210caeee111147e9904ce39a60faab42 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:59:16.314148 28749 log.cc:1196] T 210caeee111147e9904ce39a60faab42 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790679549986039-27107-0/minicluster-data/ts-0/wal/wals/210caeee111147e9904ce39a60faab42
I20260929 10:59:16.314364 28749 ts_tablet_manager.cc:1939] T 210caeee111147e9904ce39a60faab42 P 4dbb8f2d2c1d4e9e8c467c2301e71ba6: Deleting consensus metadata
I20260929 10:59:16.315070 28206 catalog_manager.cc:4841] TS 4dbb8f2d2c1d4e9e8c467c2301e71ba6 (127.26.120.193:35425): tablet 210caeee111147e9904ce39a60faab42 (table test-table [id=5a8482d33c35443d8a719b6a1e0b13a3]) successfully deleted
Sep 29 10:59:16 dist-test-slave-2lzq krb5kdc[28174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679555, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.120.193@KRBTEST.COM
I20260929 10:59:16.321393 28721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:16.310379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33578 (local address 127.26.120.193:35425)
0929 10:59:16.310418 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:16.310420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:16.310772 (+ 352us) server_negotiation.cc:407] Connection header received
0929 10:59:16.310873 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:16.310874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:16.310884 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:16.310913 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:16.311851 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:16.312615 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:16.314206 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:16.314520 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:16.314559 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:16.318226 (+ 3667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:16.318237 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:16.318237 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:16.318245 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:16.320205 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:16.320548 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:16.320552 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:16.320553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:16.320619 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:16.320824 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:16.320825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:16.320825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:16.320997 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:16.321173 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:16.321215 (+ 42us) server_negotiation.cc:299] Negotiation successful
0929 10:59:16.321297 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
W20260929 10:59:16.321718 28416 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33578
I20260929 10:59:16.329169 28763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:16.322742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36848 (local address 127.26.120.254:46489)
0929 10:59:16.322797 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:16.322799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:16.322854 (+ 55us) server_negotiation.cc:407] Connection header received
0929 10:59:16.322936 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:16.322937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:16.322948 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:16.322998 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:16.323475 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:16.324259 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:16.324923 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:16.325222 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:16.325294 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:16.326490 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:16.326492 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:16.326493 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:16.326500 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:16.327969 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:16.328342 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:16.328343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:16.328343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:16.328402 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:16.328607 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:16.328608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:16.328608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:16.328743 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:16.328845 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:16.328978 (+ 133us) server_negotiation.cc:299] Negotiation successful
0929 10:59:16.329066 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260929 10:59:16.329365 28216 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36848
I20260929 10:59:16.330046 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 28291
I20260929 10:59:16.337773 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 28431
I20260929 10:59:16.342624 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 28571
I20260929 10:59:16.347097 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 28187
2026-09-29T10:59:16Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2570 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:59:16 dist-test-slave-2lzq krb5kdc[28789](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:59:16 dist-test-slave-2lzq krb5kdc[28789](info): set up 2 sockets
Sep 29 10:59:16 dist-test-slave-2lzq krb5kdc[28789](info): commencing operation
krb5kdc: starting...
W20260929 10:59:17.397922 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 10:59:17 dist-test-slave-2lzq krb5kdc[28789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:59:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:59:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.120.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
I20260929 10:59:17.514861 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:42121
--webserver_interface=127.26.120.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:37521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:42121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:17.594663 28802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:17.594910 28802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:17.594949 28802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:17.596093 28802 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:59:17.596163 28802 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 10:59:17.596184 28802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:17.596206 28802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:17.596233 28802 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:59:17.596256 28802 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:59:17.597724 28802 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:37521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:42121
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:42121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:17.598006 28802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:17.598254 28802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:17.602927 28810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:17.603014 28808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:17.603070 28802 server_base.cc:1034] running on GCE node
W20260929 10:59:17.602954 28807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:17.603711 28802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:17.604043 28802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:17.605196 28802 hybrid_clock.cc:648] HybridClock initialized: now 1790679557605185 us; error 26 us; skew 500 ppm
Sep 29 10:59:17 dist-test-slave-2lzq krb5kdc[28789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679557, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:17.607681 28802 init.cc:377] Logged in from keytab as kudu/127.26.120.254@KRBTEST.COM (short username kudu)
I20260929 10:59:17.608109 28802 webserver.cc:460] Webserver started at http://127.26.120.254:34681/ using document root <none> and password file <none>
I20260929 10:59:17.608338 28802 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:17.608407 28802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:17.608561 28802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:17.609597 28802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "9ffa6da9b319473ea32390a2c8410b30"
format_stamp: "Formatted at 2026-09-29 10:59:17 on dist-test-slave-2lzq"
server_key: "d1ee2a739ce495a50ce7fca33b43a840"
server_key_iv: "ff45e654b3dfb37410d5cafa2f898b38"
server_key_version: "encryptionkey@0"
I20260929 10:59:17.609958 28802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "9ffa6da9b319473ea32390a2c8410b30"
format_stamp: "Formatted at 2026-09-29 10:59:17 on dist-test-slave-2lzq"
server_key: "d1ee2a739ce495a50ce7fca33b43a840"
server_key_iv: "ff45e654b3dfb37410d5cafa2f898b38"
server_key_version: "encryptionkey@0"
I20260929 10:59:17.611304 28802 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:59:17.612015 28817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:17.612188 28802 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:59:17.612286 28802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "9ffa6da9b319473ea32390a2c8410b30"
format_stamp: "Formatted at 2026-09-29 10:59:17 on dist-test-slave-2lzq"
server_key: "d1ee2a739ce495a50ce7fca33b43a840"
server_key_iv: "ff45e654b3dfb37410d5cafa2f898b38"
server_key_version: "encryptionkey@0"
I20260929 10:59:17.612368 28802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:17.631826 28802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:17.633589 28802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:17.633754 28802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:17.638396 28802 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:42121
I20260929 10:59:17.638453 28879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:42121 every 8 connection(s)
I20260929 10:59:17.638808 28802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 10:59:17.639297 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 28802
I20260929 10:59:17.639386 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 10:59:17.639500 28880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:17.639572 27107 external_mini_cluster.cc:1428] Setting key fbc40059b6cebf8f26cdd6891169826a
I20260929 10:59:17.642618 28880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ffa6da9b319473ea32390a2c8410b30: Bootstrap starting.
I20260929 10:59:17.643309 28880 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ffa6da9b319473ea32390a2c8410b30: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:17.643596 28880 log.cc:824] T 00000000000000000000000000000000 P 9ffa6da9b319473ea32390a2c8410b30: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:17.644284 28880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ffa6da9b319473ea32390a2c8410b30: No bootstrap required, opened a new log
Sep 29 10:59:17 dist-test-slave-2lzq krb5kdc[28789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:17.646755 28880 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ffa6da9b319473ea32390a2c8410b30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ffa6da9b319473ea32390a2c8410b30" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 42121 } }
I20260929 10:59:17.646890 28880 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ffa6da9b319473ea32390a2c8410b30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:17.646914 28880 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ffa6da9b319473ea32390a2c8410b30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ffa6da9b319473ea32390a2c8410b30, State: Initialized, Role: FOLLOWER
I20260929 10:59:17.647079 28880 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ffa6da9b319473ea32390a2c8410b30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ffa6da9b319473ea32390a2c8410b30" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 42121 } }
I20260929 10:59:17.647167 28880 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ffa6da9b319473ea32390a2c8410b30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:59:17.647202 28880 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ffa6da9b319473ea32390a2c8410b30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:59:17.647253 28880 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ffa6da9b319473ea32390a2c8410b30 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:17.647915 28880 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ffa6da9b319473ea32390a2c8410b30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ffa6da9b319473ea32390a2c8410b30" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 42121 } }
I20260929 10:59:17.648048 28880 leader_election.cc:302] T 00000000000000000000000000000000 P 9ffa6da9b319473ea32390a2c8410b30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ffa6da9b319473ea32390a2c8410b30; no voters:
I20260929 10:59:17.648209 28880 leader_election.cc:288] T 00000000000000000000000000000000 P 9ffa6da9b319473ea32390a2c8410b30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:59:17.648268 28885 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ffa6da9b319473ea32390a2c8410b30 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:59:17.648414 28885 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ffa6da9b319473ea32390a2c8410b30 [term 1 LEADER]: Becoming Leader. State: Replica: 9ffa6da9b319473ea32390a2c8410b30, State: Running, Role: LEADER
I20260929 10:59:17.648497 28885 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ffa6da9b319473ea32390a2c8410b30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ffa6da9b319473ea32390a2c8410b30" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 42121 } }
I20260929 10:59:17.648890 28886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ffa6da9b319473ea32390a2c8410b30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ffa6da9b319473ea32390a2c8410b30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ffa6da9b319473ea32390a2c8410b30" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 42121 } } }
I20260929 10:59:17.648990 28886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ffa6da9b319473ea32390a2c8410b30 [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:17.648895 28887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ffa6da9b319473ea32390a2c8410b30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ffa6da9b319473ea32390a2c8410b30. Latest consensus state: current_term: 1 leader_uuid: "9ffa6da9b319473ea32390a2c8410b30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ffa6da9b319473ea32390a2c8410b30" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 42121 } } }
I20260929 10:59:17.649093 28887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ffa6da9b319473ea32390a2c8410b30 [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:17.649236 28889 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:59:17.649740 28889 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:59:17.648471 28880 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ffa6da9b319473ea32390a2c8410b30 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:59:17.651283 28883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:17.640095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47780 (local address 127.26.120.254:42121)
0929 10:59:17.640275 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:17.640284 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:17.640317 (+ 33us) server_negotiation.cc:407] Connection header received
0929 10:59:17.640898 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:17.640905 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:17.640951 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:17.641064 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:17.641838 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:17.642825 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:17.643902 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:17.644426 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:17.644508 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:17.647115 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:17.647118 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:17.647118 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:17.647129 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:17.649186 (+ 2057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:17.649578 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:17.649581 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:17.649582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:17.649684 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:17.649927 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:17.649928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:17.649929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:17.650251 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:17.650409 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:17.650897 (+ 488us) server_negotiation.cc:299] Negotiation successful
0929 10:59:17.651060 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":87,"threads_started":1}
I20260929 10:59:17.652401 28889 catalog_manager.cc:1347] Generated new cluster ID: 4d19983286ed4cb2a211130e8749a71f
I20260929 10:59:17.652462 28889 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:59:17.685595 28889 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:59:17.686404 28889 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:59:17.699608 28889 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ffa6da9b319473ea32390a2c8410b30: Generated new TSK 0
I20260929 10:59:17.699826 28889 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.120.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab.
Entry for principal kudu/127.26.120.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab.
I20260929 10:59:17.731065 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.193:0
--local_ip_for_outbound_sockets=127.26.120.193
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:42121
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:37521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:17.812280 28906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:17.812520 28906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:17.812563 28906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:17.813745 28906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:17.813814 28906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:17.813859 28906 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 10:59:17.813915 28906 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 10:59:17.813954 28906 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.193
I20260929 10:59:17.815656 28906 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:37521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.120.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.120.254:42121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28906
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.193
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:17.815925 28906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:17.816182 28906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:17.821132 28911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:17.821142 28912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:17.821142 28914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:17.821312 28906 server_base.cc:1034] running on GCE node
I20260929 10:59:17.821559 28906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:17.821789 28906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:17.822947 28906 hybrid_clock.cc:648] HybridClock initialized: now 1790679557822942 us; error 39 us; skew 500 ppm
Sep 29 10:59:17 dist-test-slave-2lzq krb5kdc[28789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679557, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:17.825392 28906 init.cc:377] Logged in from keytab as kudu/127.26.120.193@KRBTEST.COM (short username kudu)
I20260929 10:59:17.825881 28906 webserver.cc:460] Webserver started at http://127.26.120.193:35335/ using document root <none> and password file <none>
I20260929 10:59:17.826191 28906 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:17.826260 28906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:17.826398 28906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:17.827430 28906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-0/data/instance:
uuid: "cc42405df36e42629f0e2fd1a75c2751"
format_stamp: "Formatted at 2026-09-29 10:59:17 on dist-test-slave-2lzq"
server_key: "267462e054117455b0241fc4c45ed1c3"
server_key_iv: "62bdf818e454dfe7aeb18e9f49aabf37"
server_key_version: "encryptionkey@0"
I20260929 10:59:17.827793 28906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance:
uuid: "cc42405df36e42629f0e2fd1a75c2751"
format_stamp: "Formatted at 2026-09-29 10:59:17 on dist-test-slave-2lzq"
server_key: "267462e054117455b0241fc4c45ed1c3"
server_key_iv: "62bdf818e454dfe7aeb18e9f49aabf37"
server_key_version: "encryptionkey@0"
I20260929 10:59:17.829108 28906 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 10:59:17.829808 28921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:17.829985 28906 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:59:17.830116 28906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-0/wal
uuid: "cc42405df36e42629f0e2fd1a75c2751"
format_stamp: "Formatted at 2026-09-29 10:59:17 on dist-test-slave-2lzq"
server_key: "267462e054117455b0241fc4c45ed1c3"
server_key_iv: "62bdf818e454dfe7aeb18e9f49aabf37"
server_key_version: "encryptionkey@0"
I20260929 10:59:17.830204 28906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:17.841287 28906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:17.843079 28906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:17.843235 28906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:17.843768 28906 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:17.843832 28906 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:17.843878 28906 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:17.843896 28906 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 10:59:17 dist-test-slave-2lzq krb5kdc[28789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679557, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:17.851052 28906 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.193:34187
I20260929 10:59:17.851387 28906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
I20260929 10:59:17.851779 29036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.193:34187 every 8 connection(s)
I20260929 10:59:17.853324 28883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:17.844142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:45319 (local address 127.26.120.254:42121)
0929 10:59:17.844208 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:17.844210 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:17.844452 (+ 242us) server_negotiation.cc:407] Connection header received
0929 10:59:17.844974 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:17.844975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:17.844988 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:17.845043 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:17.845677 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:17.846368 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:17.847373 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:17.847593 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:17.847671 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:17.850444 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:17.850446 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:17.850446 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:17.850454 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:17.852120 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:17.852590 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:17.852591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:17.852591 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:17.852642 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:17.852840 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:17.852841 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:17.852841 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:17.852932 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:17.852979 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:17.853195 (+ 216us) server_negotiation.cc:299] Negotiation successful
0929 10:59:17.853262 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 10:59:17.853497 28930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:17.844146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:42121 (local address 127.26.120.193:45319)
0929 10:59:17.844319 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:17.844359 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:17.844862 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:17.845112 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:17.845116 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:17.845398 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:17.845579 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:17.845584 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:17.846484 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:17.846485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:17.847257 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:17.847259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:17.847678 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:17.847679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:17.847799 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:17.848308 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:17.848317 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:17.850286 (+ 1969us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:17.852193 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:17.852197 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:17.852199 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:17.852517 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:17.852703 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:17.852704 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:17.852704 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:17.852797 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:17.852981 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:17.852985 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:17.853093 (+ 108us) client_negotiation.cc:769] Sending connection context
0929 10:59:17.853171 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 10:59:17.853282 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":97,"threads_started":1}
I20260929 10:59:17.853999 29037 heartbeater.cc:347] Connected to a master server at 127.26.120.254:42121
I20260929 10:59:17.854095 29037 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:17.854277 29037 heartbeater.cc:511] Master 127.26.120.254:42121 requested a full tablet report, sending...
I20260929 10:59:17.854800 28833 ts_manager.cc:194] Registered new tserver with Master: cc42405df36e42629f0e2fd1a75c2751 (127.26.120.193:34187)
I20260929 10:59:17.855736 28833 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.193@KRBTEST.COM'} at 127.26.120.193:45319
I20260929 10:59:17.856115 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 28906
I20260929 10:59:17.856205 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance
I20260929 10:59:17.856401 27107 external_mini_cluster.cc:1428] Setting key 0c5e48ca7e3b5e7f9a0e35eeee74fbe9
I20260929 10:59:17.861014 28883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:17.855113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:56143 (local address 127.26.120.254:42121)
0929 10:59:17.855148 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:17.855150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:17.855210 (+ 60us) server_negotiation.cc:407] Connection header received
0929 10:59:17.855271 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:17.855271 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:17.855279 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:17.855322 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:17.855639 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:17.856315 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:17.857222 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:17.857530 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:17.857598 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:17.858505 (+ 907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:17.858507 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:17.858507 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:17.858515 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:17.859830 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:17.860230 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:17.860230 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:17.860231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:17.860277 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:17.860472 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:17.860473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:17.860473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:17.860563 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:17.860610 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:17.860867 (+ 257us) server_negotiation.cc:299] Negotiation successful
0929 10:59:17.860957 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 10:59:17.861045 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:17.855028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:42121 (local address 127.26.120.193:56143)
0929 10:59:17.855154 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:17.855163 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:17.855218 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:17.855334 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:17.855334 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:17.855515 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:17.855579 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:17.855580 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:17.856428 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:17.856429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:17.857163 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:17.857165 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:17.857610 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:17.857611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:17.857667 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:17.858108 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:17.858113 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:17.858396 (+ 283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:17.859913 (+ 1517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:17.859914 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:17.859915 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:17.860158 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:17.860340 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:17.860341 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:17.860341 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:17.860414 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:17.860626 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:17.860627 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:17.860689 (+ 62us) client_negotiation.cc:769] Sending connection context
0929 10:59:17.860768 (+ 79us) client_negotiation.cc:240] Negotiation successful
0929 10:59:17.860881 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":66,"threads_started":1}
WARNING: no policy specified for kudu/127.26.120.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab.
Entry for principal kudu/127.26.120.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab.
I20260929 10:59:17.881899 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.194:0
--local_ip_for_outbound_sockets=127.26.120.194
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:42121
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:37521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:17.961696 29046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:17.961894 29046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:17.961917 29046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:17.963090 29046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:17.963133 29046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:17.963153 29046 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 10:59:17.963165 29046 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 10:59:17.963182 29046 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.194
I20260929 10:59:17.964555 29046 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:37521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.120.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.120.254:42121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.194
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:17.964745 29046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:17.964915 29046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:17.969810 29052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:17.969830 29051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:17.969827 29054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:17.969847 29046 server_base.cc:1034] running on GCE node
I20260929 10:59:17.970230 29046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:17.970517 29046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:17.971695 29046 hybrid_clock.cc:648] HybridClock initialized: now 1790679557971673 us; error 39 us; skew 500 ppm
Sep 29 10:59:17 dist-test-slave-2lzq krb5kdc[28789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679557, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:17.974267 29046 init.cc:377] Logged in from keytab as kudu/127.26.120.194@KRBTEST.COM (short username kudu)
I20260929 10:59:17.974659 29046 webserver.cc:460] Webserver started at http://127.26.120.194:36249/ using document root <none> and password file <none>
I20260929 10:59:17.974853 29046 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:17.974895 29046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:17.974983 29046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:17.976019 29046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-1/data/instance:
uuid: "9f3290ac12b64068a3316deb116f06f8"
format_stamp: "Formatted at 2026-09-29 10:59:17 on dist-test-slave-2lzq"
server_key: "d66536af730cb94df0d6e09745939cd0"
server_key_iv: "843591324bd2ad8209b9735a8f698ea3"
server_key_version: "encryptionkey@0"
I20260929 10:59:17.976390 29046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance:
uuid: "9f3290ac12b64068a3316deb116f06f8"
format_stamp: "Formatted at 2026-09-29 10:59:17 on dist-test-slave-2lzq"
server_key: "d66536af730cb94df0d6e09745939cd0"
server_key_iv: "843591324bd2ad8209b9735a8f698ea3"
server_key_version: "encryptionkey@0"
I20260929 10:59:17.977792 29046 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 10:59:17.978622 29061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:17.978801 29046 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:17.978906 29046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-1/wal
uuid: "9f3290ac12b64068a3316deb116f06f8"
format_stamp: "Formatted at 2026-09-29 10:59:17 on dist-test-slave-2lzq"
server_key: "d66536af730cb94df0d6e09745939cd0"
server_key_iv: "843591324bd2ad8209b9735a8f698ea3"
server_key_version: "encryptionkey@0"
I20260929 10:59:17.978989 29046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:17.995289 29046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:17.997673 29046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:17.997835 29046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:17.998482 29046 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:17.998548 29046 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:17.998584 29046 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:17.998610 29046 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 10:59:17 dist-test-slave-2lzq krb5kdc[28789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679557, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:18.005685 29046 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.194:41721
I20260929 10:59:18.005789 29176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.194:41721 every 8 connection(s)
I20260929 10:59:18.006112 29046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
I20260929 10:59:18.006739 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 29046
I20260929 10:59:18.006846 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance
I20260929 10:59:18.007054 27107 external_mini_cluster.cc:1428] Setting key fc4f1c8559269367dafccabd6fb9b6fa
I20260929 10:59:18.009155 28883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:17.998974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:44001 (local address 127.26.120.254:42121)
0929 10:59:17.999052 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:17.999056 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:17.999263 (+ 207us) server_negotiation.cc:407] Connection header received
0929 10:59:17.999860 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:17.999862 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:17.999877 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:17.999969 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:18.000867 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.001719 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.002869 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.003225 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.003313 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.005994 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:18.005996 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:18.005996 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:18.006004 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:18.007732 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:18.008151 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:18.008153 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:18.008153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:18.008218 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:18.008430 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:18.008431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:18.008431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:18.008575 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:18.008666 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:18.008938 (+ 272us) server_negotiation.cc:299] Negotiation successful
0929 10:59:18.009027 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 10:59:18.009198 29070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:17.998970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:42121 (local address 127.26.120.194:44001)
0929 10:59:17.999125 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:17.999165 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:17.999732 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:18.000027 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:18.000031 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:18.000446 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:18.000751 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:18.000757 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.001815 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.001816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:18.002763 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:18.002765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.003347 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.003348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:18.003510 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.004028 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:18.004037 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:18.005879 (+ 1842us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:18.007853 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:18.007858 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:18.007860 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:18.008074 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:18.008282 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:18.008282 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:18.008283 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:18.008370 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:18.008658 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:18.008662 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:18.008809 (+ 147us) client_negotiation.cc:769] Sending connection context
0929 10:59:18.008896 (+ 87us) client_negotiation.cc:240] Negotiation successful
0929 10:59:18.009014 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
I20260929 10:59:18.009682 29177 heartbeater.cc:347] Connected to a master server at 127.26.120.254:42121
I20260929 10:59:18.009768 29177 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:18.009948 29177 heartbeater.cc:511] Master 127.26.120.254:42121 requested a full tablet report, sending...
I20260929 10:59:18.010285 28834 ts_manager.cc:194] Registered new tserver with Master: 9f3290ac12b64068a3316deb116f06f8 (127.26.120.194:41721)
I20260929 10:59:18.010718 28834 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.194@KRBTEST.COM'} at 127.26.120.194:44001
WARNING: no policy specified for kudu/127.26.120.195@KRBTEST.COM; defaulting to no policy
I20260929 10:59:18.017050 29185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:18.010830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:42121 (local address 127.26.120.194:35963)
0929 10:59:18.011142 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:18.011152 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:18.011211 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:18.011353 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:18.011354 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:18.011531 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:18.011594 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:18.011596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.012586 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.012587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:18.013075 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:18.013076 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.013583 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.013583 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:18.013635 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.014085 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:18.014090 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:18.014466 (+ 376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:18.016048 (+ 1582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:18.016049 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:18.016050 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:18.016216 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:18.016433 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:18.016434 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:18.016434 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:18.016484 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:18.016753 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:18.016754 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:18.016804 (+ 50us) client_negotiation.cc:769] Sending connection context
0929 10:59:18.016899 (+ 95us) client_negotiation.cc:240] Negotiation successful
0929 10:59:18.016979 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":73,"threads_started":1}
I20260929 10:59:18.017051 28883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:18.010847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:35963 (local address 127.26.120.254:42121)
0929 10:59:18.010889 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:18.010891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:18.011220 (+ 329us) server_negotiation.cc:407] Connection header received
0929 10:59:18.011263 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:18.011264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:18.011275 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:18.011345 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:18.011674 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.012493 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.013157 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.013502 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.013572 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.014553 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:18.014555 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:18.014555 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:18.014563 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:18.015947 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:18.016295 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:18.016296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:18.016296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:18.016360 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:18.016553 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:18.016554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:18.016554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:18.016687 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:18.016742 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:18.016933 (+ 191us) server_negotiation.cc:299] Negotiation successful
0929 10:59:18.016994 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab.
Entry for principal kudu/127.26.120.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab.
I20260929 10:59:18.032743 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.195:0
--local_ip_for_outbound_sockets=127.26.120.195
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:42121
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:37521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:18.112298 29187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:18.112499 29187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:18.112524 29187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:18.113580 29187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:18.113615 29187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:18.113636 29187 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 10:59:18.113646 29187 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 10:59:18.113663 29187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.195
I20260929 10:59:18.115077 29187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:37521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.120.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.120.254:42121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.195
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:18.115324 29187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:18.115633 29187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:18.121327 29195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:18.121340 29192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:18.121351 29193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:18.121693 29187 server_base.cc:1034] running on GCE node
I20260929 10:59:18.121904 29187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:18.122167 29187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:18.123327 29187 hybrid_clock.cc:648] HybridClock initialized: now 1790679558123322 us; error 36 us; skew 500 ppm
Sep 29 10:59:18 dist-test-slave-2lzq krb5kdc[28789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679558, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:18.125778 29187 init.cc:377] Logged in from keytab as kudu/127.26.120.195@KRBTEST.COM (short username kudu)
I20260929 10:59:18.126284 29187 webserver.cc:460] Webserver started at http://127.26.120.195:34989/ using document root <none> and password file <none>
I20260929 10:59:18.126492 29187 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:18.126534 29187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:18.126621 29187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:18.127580 29187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-2/data/instance:
uuid: "2bffd7b80ee6450fa765d1ed59a558c0"
format_stamp: "Formatted at 2026-09-29 10:59:18 on dist-test-slave-2lzq"
server_key: "e793c57a1c9a811842407a51f40766f2"
server_key_iv: "1b91de609a526dfa03104abe4300693c"
server_key_version: "encryptionkey@0"
I20260929 10:59:18.127871 29187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance:
uuid: "2bffd7b80ee6450fa765d1ed59a558c0"
format_stamp: "Formatted at 2026-09-29 10:59:18 on dist-test-slave-2lzq"
server_key: "e793c57a1c9a811842407a51f40766f2"
server_key_iv: "1b91de609a526dfa03104abe4300693c"
server_key_version: "encryptionkey@0"
I20260929 10:59:18.129088 29187 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:59:18.129741 29202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:18.129901 29187 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:59:18.129987 29187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-2/wal
uuid: "2bffd7b80ee6450fa765d1ed59a558c0"
format_stamp: "Formatted at 2026-09-29 10:59:18 on dist-test-slave-2lzq"
server_key: "e793c57a1c9a811842407a51f40766f2"
server_key_iv: "1b91de609a526dfa03104abe4300693c"
server_key_version: "encryptionkey@0"
I20260929 10:59:18.130043 29187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:18.141570 29187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:18.143811 29187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:18.143922 29187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:18.144354 29187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:18.144387 29187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:18.144418 29187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:18.144456 29187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 10:59:18 dist-test-slave-2lzq krb5kdc[28789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679558, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:18.152019 29187 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.195:44043
I20260929 10:59:18.152086 29318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.195:44043 every 8 connection(s)
I20260929 10:59:18.152375 29187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
I20260929 10:59:18.155189 28883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:18.144972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:50873 (local address 127.26.120.254:42121)
0929 10:59:18.145015 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:18.145020 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:18.145262 (+ 242us) server_negotiation.cc:407] Connection header received
0929 10:59:18.145874 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:18.145876 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:18.145889 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:18.145974 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:18.146940 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.147830 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.148945 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.149310 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.149394 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.152362 (+ 2968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:18.152364 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:18.152364 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:18.152373 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:18.153846 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:18.154343 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:18.154344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:18.154345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:18.154398 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:18.154701 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:18.154701 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:18.154701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:18.154804 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:18.154851 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:18.155051 (+ 200us) server_negotiation.cc:299] Negotiation successful
0929 10:59:18.155109 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 10:59:18.155412 29211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:18.144977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:42121 (local address 127.26.120.195:50873)
0929 10:59:18.145135 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:18.145171 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:18.145729 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:18.146051 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:18.146079 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:18.146485 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:18.146815 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:18.146822 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.147934 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.147935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:18.148831 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:18.148834 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.149416 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.149417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:18.149583 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.150177 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:18.150185 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:18.152234 (+ 2049us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:18.153919 (+ 1685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:18.153924 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:18.153926 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:18.154247 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:18.154510 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:18.154511 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:18.154511 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:18.154633 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:18.154844 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:18.154847 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:18.154961 (+ 114us) client_negotiation.cc:769] Sending connection context
0929 10:59:18.155034 (+ 73us) client_negotiation.cc:240] Negotiation successful
0929 10:59:18.155154 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":93,"threads_started":1}
I20260929 10:59:18.155913 29319 heartbeater.cc:347] Connected to a master server at 127.26.120.254:42121
I20260929 10:59:18.156002 29319 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:18.156204 29319 heartbeater.cc:511] Master 127.26.120.254:42121 requested a full tablet report, sending...
I20260929 10:59:18.156610 28833 ts_manager.cc:194] Registered new tserver with Master: 2bffd7b80ee6450fa765d1ed59a558c0 (127.26.120.195:44043)
I20260929 10:59:18.157052 28833 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.195@KRBTEST.COM'} at 127.26.120.195:50873
I20260929 10:59:18.157582 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 29187
I20260929 10:59:18.157670 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance
I20260929 10:59:18.157856 27107 external_mini_cluster.cc:1428] Setting key cdb9ef5036b0ab32686a507bde2d4cd8
I20260929 10:59:18.158627 27107 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:59:18.163448 28883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:18.157128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:53495 (local address 127.26.120.254:42121)
0929 10:59:18.157182 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:18.157185 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:18.157296 (+ 111us) server_negotiation.cc:407] Connection header received
0929 10:59:18.157330 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:18.157331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:18.157342 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:18.157445 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:18.157823 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.158657 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.159448 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.159666 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.159756 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.160660 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:18.160661 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:18.160661 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:18.160670 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:18.162217 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:18.162612 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:18.162612 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:18.162613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:18.162663 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:18.162892 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:18.162893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:18.162894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:18.163014 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:18.163097 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:18.163268 (+ 171us) server_negotiation.cc:299] Negotiation successful
0929 10:59:18.163349 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260929 10:59:18.163457 29325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:18.157046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:42121 (local address 127.26.120.195:53495)
0929 10:59:18.157212 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:18.157222 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:18.157275 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:18.157483 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:18.157484 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:18.157656 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:18.157745 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:18.157747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.158799 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.158800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:18.159345 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:18.159347 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.159757 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.159758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:18.159839 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.160306 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:18.160310 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:18.160569 (+ 259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:18.162292 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:18.162293 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:18.162293 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:18.162534 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:18.162740 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:18.162741 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:18.162742 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:18.162818 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:18.163089 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:18.163090 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:18.163139 (+ 49us) client_negotiation.cc:769] Sending connection context
0929 10:59:18.163236 (+ 97us) client_negotiation.cc:240] Negotiation successful
0929 10:59:18.163351 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":100,"threads_started":1}
Sep 29 10:59:18 dist-test-slave-2lzq krb5kdc[28789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 10:59:18 dist-test-slave-2lzq krb5kdc[28789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:18.181905 28883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:18.175067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47786 (local address 127.26.120.254:42121)
0929 10:59:18.175153 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:18.175155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:18.175183 (+ 28us) server_negotiation.cc:407] Connection header received
0929 10:59:18.175205 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:18.175206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:18.175215 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:18.175264 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:18.175855 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.176351 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.177116 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.177318 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.177382 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.179505 (+ 2123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:18.179506 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:18.179506 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:18.179513 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:18.180818 (+ 1305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:18.181149 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:18.181149 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:18.181150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:18.181199 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:18.181407 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:18.181407 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:18.181408 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:18.181508 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:18.181581 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:18.181765 (+ 184us) server_negotiation.cc:299] Negotiation successful
0929 10:59:18.181823 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 10:59:18.183415 28833 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47786:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:59:18.184988 28833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:59:18.191337 29333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:18.187153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:34187 (local address 127.0.0.1:60498)
0929 10:59:18.187318 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:18.187356 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:18.187422 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:18.187873 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:18.187873 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:18.187875 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:18.187933 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:18.187934 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.189253 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.189254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:18.190333 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:18.190335 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.191024 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.191025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:18.191115 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.191129 (+ 14us) client_negotiation.cc:769] Sending connection context
0929 10:59:18.191199 (+ 70us) client_negotiation.cc:240] Negotiation successful
0929 10:59:18.191258 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":18,"thread_start_us":76,"threads_started":1}
I20260929 10:59:18.191425 29339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:18.187412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46918 (local address 127.26.120.194:41721)
0929 10:59:18.187803 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:18.187808 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:18.187819 (+ 11us) server_negotiation.cc:407] Connection header received
0929 10:59:18.187879 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:18.187880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:18.187894 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:18.188103 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:18.188170 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.189270 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.190421 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.191097 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.191164 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.191211 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:18.191290 (+ 79us) server_negotiation.cc:299] Negotiation successful
0929 10:59:18.191328 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":77,"threads_started":1}
I20260929 10:59:18.191597 29337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:18.187239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60498 (local address 127.26.120.193:34187)
0929 10:59:18.187424 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:18.187429 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:18.187440 (+ 11us) server_negotiation.cc:407] Connection header received
0929 10:59:18.187631 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:18.187634 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:18.187653 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:18.187743 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:18.188012 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.189160 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.190418 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.190910 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.191003 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.191060 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:18.191464 (+ 404us) server_negotiation.cc:299] Negotiation successful
0929 10:59:18.191506 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":83,"threads_started":1}
I20260929 10:59:18.191425 29336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:18.187176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:41721 (local address 127.0.0.1:46918)
0929 10:59:18.187646 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:18.187654 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:18.187691 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:18.187992 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:18.187993 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:18.187995 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:18.188033 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:18.188034 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.189338 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.189339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:18.190327 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:18.190330 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.191162 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.191163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:18.191222 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.191225 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:59:18.191276 (+ 51us) client_negotiation.cc:240] Negotiation successful
0929 10:59:18.191310 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":57,"threads_started":1}
I20260929 10:59:18.191812 29335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:18.187153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:44043 (local address 127.0.0.1:45592)
0929 10:59:18.187490 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:18.187500 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:18.187547 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:18.187764 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:18.187767 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:18.187770 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:18.187878 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:18.187880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.188879 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.188880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:18.190152 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:18.190155 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.191153 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.191155 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:18.191228 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.191232 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:59:18.191281 (+ 49us) client_negotiation.cc:240] Negotiation successful
0929 10:59:18.191336 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":27,"thread_start_us":54,"threads_started":1}
I20260929 10:59:18.192282 29111 tablet_service.cc:1467] Processing CreateTablet for tablet c1c0efaa113e406283b95025e32fc01d (DEFAULT_TABLE table=test-table [id=c59c0a8921c64c6cb8aa5f4fb3969edb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:18.192283 28971 tablet_service.cc:1467] Processing CreateTablet for tablet c1c0efaa113e406283b95025e32fc01d (DEFAULT_TABLE table=test-table [id=c59c0a8921c64c6cb8aa5f4fb3969edb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:18.192638 28971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1c0efaa113e406283b95025e32fc01d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:18.192638 29111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1c0efaa113e406283b95025e32fc01d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:18.194228 29338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:18.187391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45592 (local address 127.26.120.195:44043)
0929 10:59:18.187589 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:18.187594 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:18.187606 (+ 12us) server_negotiation.cc:407] Connection header received
0929 10:59:18.187643 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:18.187645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:18.187660 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:18.187744 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:18.187984 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.188773 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.190236 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.190990 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.191064 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.191117 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:18.191378 (+ 261us) server_negotiation.cc:299] Negotiation successful
0929 10:59:18.194155 (+ 2777us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":82,"threads_started":1}
I20260929 10:59:18.194756 29252 tablet_service.cc:1467] Processing CreateTablet for tablet c1c0efaa113e406283b95025e32fc01d (DEFAULT_TABLE table=test-table [id=c59c0a8921c64c6cb8aa5f4fb3969edb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:18.195091 29252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1c0efaa113e406283b95025e32fc01d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:18.196794 29341 tablet_bootstrap.cc:492] T c1c0efaa113e406283b95025e32fc01d P 9f3290ac12b64068a3316deb116f06f8: Bootstrap starting.
I20260929 10:59:18.197005 29340 tablet_bootstrap.cc:492] T c1c0efaa113e406283b95025e32fc01d P cc42405df36e42629f0e2fd1a75c2751: Bootstrap starting.
I20260929 10:59:18.197692 29340 tablet_bootstrap.cc:654] T c1c0efaa113e406283b95025e32fc01d P cc42405df36e42629f0e2fd1a75c2751: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:18.197953 29342 tablet_bootstrap.cc:492] T c1c0efaa113e406283b95025e32fc01d P 2bffd7b80ee6450fa765d1ed59a558c0: Bootstrap starting.
I20260929 10:59:18.198076 29340 log.cc:824] T c1c0efaa113e406283b95025e32fc01d P cc42405df36e42629f0e2fd1a75c2751: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:18.198899 29341 tablet_bootstrap.cc:654] T c1c0efaa113e406283b95025e32fc01d P 9f3290ac12b64068a3316deb116f06f8: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:18.199002 29340 tablet_bootstrap.cc:492] T c1c0efaa113e406283b95025e32fc01d P cc42405df36e42629f0e2fd1a75c2751: No bootstrap required, opened a new log
I20260929 10:59:18.199095 29340 ts_tablet_manager.cc:1397] T c1c0efaa113e406283b95025e32fc01d P cc42405df36e42629f0e2fd1a75c2751: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:59:18.199111 29342 tablet_bootstrap.cc:654] T c1c0efaa113e406283b95025e32fc01d P 2bffd7b80ee6450fa765d1ed59a558c0: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:18.199255 29341 log.cc:824] T c1c0efaa113e406283b95025e32fc01d P 9f3290ac12b64068a3316deb116f06f8: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:18.199447 29342 log.cc:824] T c1c0efaa113e406283b95025e32fc01d P 2bffd7b80ee6450fa765d1ed59a558c0: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:18.200093 29341 tablet_bootstrap.cc:492] T c1c0efaa113e406283b95025e32fc01d P 9f3290ac12b64068a3316deb116f06f8: No bootstrap required, opened a new log
I20260929 10:59:18.200172 29341 ts_tablet_manager.cc:1397] T c1c0efaa113e406283b95025e32fc01d P 9f3290ac12b64068a3316deb116f06f8: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260929 10:59:18.200220 29342 tablet_bootstrap.cc:492] T c1c0efaa113e406283b95025e32fc01d P 2bffd7b80ee6450fa765d1ed59a558c0: No bootstrap required, opened a new log
I20260929 10:59:18.200287 29342 ts_tablet_manager.cc:1397] T c1c0efaa113e406283b95025e32fc01d P 2bffd7b80ee6450fa765d1ed59a558c0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:59:18.201437 29340 raft_consensus.cc:348] T c1c0efaa113e406283b95025e32fc01d P cc42405df36e42629f0e2fd1a75c2751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } } peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } }
I20260929 10:59:18.201628 29340 raft_consensus.cc:374] T c1c0efaa113e406283b95025e32fc01d P cc42405df36e42629f0e2fd1a75c2751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:18.201670 29340 raft_consensus.cc:729] T c1c0efaa113e406283b95025e32fc01d P cc42405df36e42629f0e2fd1a75c2751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc42405df36e42629f0e2fd1a75c2751, State: Initialized, Role: FOLLOWER
I20260929 10:59:18.201617 29342 raft_consensus.cc:348] T c1c0efaa113e406283b95025e32fc01d P 2bffd7b80ee6450fa765d1ed59a558c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } } peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } }
I20260929 10:59:18.201807 29342 raft_consensus.cc:374] T c1c0efaa113e406283b95025e32fc01d P 2bffd7b80ee6450fa765d1ed59a558c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:18.201848 29342 raft_consensus.cc:729] T c1c0efaa113e406283b95025e32fc01d P 2bffd7b80ee6450fa765d1ed59a558c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bffd7b80ee6450fa765d1ed59a558c0, State: Initialized, Role: FOLLOWER
I20260929 10:59:18.201824 29340 consensus_queue.cc:260] T c1c0efaa113e406283b95025e32fc01d P cc42405df36e42629f0e2fd1a75c2751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } } peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } }
I20260929 10:59:18.201980 29342 consensus_queue.cc:260] T c1c0efaa113e406283b95025e32fc01d P 2bffd7b80ee6450fa765d1ed59a558c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } } peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } }
I20260929 10:59:18.202153 29340 ts_tablet_manager.cc:1428] T c1c0efaa113e406283b95025e32fc01d P cc42405df36e42629f0e2fd1a75c2751: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 10:59:18.202250 29342 ts_tablet_manager.cc:1428] T c1c0efaa113e406283b95025e32fc01d P 2bffd7b80ee6450fa765d1ed59a558c0: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260929 10:59:18.202275 29037 heartbeater.cc:503] Master 127.26.120.254:42121 was elected leader, sending a full tablet report...
I20260929 10:59:18.202436 29341 raft_consensus.cc:348] T c1c0efaa113e406283b95025e32fc01d P 9f3290ac12b64068a3316deb116f06f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } } peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } }
I20260929 10:59:18.202598 29341 raft_consensus.cc:374] T c1c0efaa113e406283b95025e32fc01d P 9f3290ac12b64068a3316deb116f06f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:18.202636 29341 raft_consensus.cc:729] T c1c0efaa113e406283b95025e32fc01d P 9f3290ac12b64068a3316deb116f06f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f3290ac12b64068a3316deb116f06f8, State: Initialized, Role: FOLLOWER
I20260929 10:59:18.202746 29341 consensus_queue.cc:260] T c1c0efaa113e406283b95025e32fc01d P 9f3290ac12b64068a3316deb116f06f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } } peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } }
I20260929 10:59:18.203045 29341 ts_tablet_manager.cc:1428] T c1c0efaa113e406283b95025e32fc01d P 9f3290ac12b64068a3316deb116f06f8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 10:59:18.203099 29177 heartbeater.cc:503] Master 127.26.120.254:42121 was elected leader, sending a full tablet report...
I20260929 10:59:18.203166 29319 heartbeater.cc:503] Master 127.26.120.254:42121 was elected leader, sending a full tablet report...
I20260929 10:59:18.221470 29348 raft_consensus.cc:482] T c1c0efaa113e406283b95025e32fc01d P 9f3290ac12b64068a3316deb116f06f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:59:18.221563 29348 raft_consensus.cc:504] T c1c0efaa113e406283b95025e32fc01d P 9f3290ac12b64068a3316deb116f06f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } } peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } }
I20260929 10:59:18.221930 29348 leader_election.cc:288] T c1c0efaa113e406283b95025e32fc01d P 9f3290ac12b64068a3316deb116f06f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2bffd7b80ee6450fa765d1ed59a558c0 (127.26.120.195:44043), cc42405df36e42629f0e2fd1a75c2751 (127.26.120.193:34187)
I20260929 10:59:18.224457 29346 raft_consensus.cc:482] T c1c0efaa113e406283b95025e32fc01d P cc42405df36e42629f0e2fd1a75c2751 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:59:18.224581 29346 raft_consensus.cc:504] T c1c0efaa113e406283b95025e32fc01d P cc42405df36e42629f0e2fd1a75c2751 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } } peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } }
I20260929 10:59:18.224916 29346 leader_election.cc:288] T c1c0efaa113e406283b95025e32fc01d P cc42405df36e42629f0e2fd1a75c2751 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2bffd7b80ee6450fa765d1ed59a558c0 (127.26.120.195:44043), 9f3290ac12b64068a3316deb116f06f8 (127.26.120.194:41721)
I20260929 10:59:18.225140 29070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:18.222180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:34187 (local address 127.26.120.194:41259)
0929 10:59:18.222250 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:18.222263 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:18.222308 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:18.222470 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:18.222471 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:18.222473 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:18.222557 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:18.222568 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.223377 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.223378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:18.224154 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:18.224156 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.224863 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.224864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:18.224951 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.224955 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:59:18.225004 (+ 49us) client_negotiation.cc:240] Negotiation successful
0929 10:59:18.225042 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260929 10:59:18.225243 29338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:18.222272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:37523 (local address 127.26.120.195:44043)
0929 10:59:18.222317 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:18.222319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:18.222567 (+ 248us) server_negotiation.cc:407] Connection header received
0929 10:59:18.222607 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:18.222608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:18.222619 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:18.222678 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:18.222869 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.223388 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.224326 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.224883 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.224946 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.224953 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:18.225132 (+ 179us) server_negotiation.cc:299] Negotiation successful
0929 10:59:18.225172 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260929 10:59:18.225323 29337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:18.222279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:41259 (local address 127.26.120.193:34187)
0929 10:59:18.222324 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:18.222327 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:18.222335 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:59:18.222364 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:18.222364 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:18.222374 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:18.222432 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:18.222640 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.223270 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.224247 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.224767 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.224837 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.224845 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:18.225212 (+ 367us) server_negotiation.cc:299] Negotiation successful
0929 10:59:18.225256 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 10:59:18.225720 28991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1c0efaa113e406283b95025e32fc01d" candidate_uuid: "9f3290ac12b64068a3316deb116f06f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc42405df36e42629f0e2fd1a75c2751" is_pre_election: true
I20260929 10:59:18.225831 29349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:18.222181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:44043 (local address 127.26.120.194:37523)
0929 10:59:18.222488 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:18.222498 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:18.222554 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:18.222691 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:18.222691 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:18.222693 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:18.222770 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:18.222771 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.223465 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.223465 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:18.224184 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:18.224185 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.224983 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.224984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:18.225054 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.225057 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:59:18.225696 (+ 639us) client_negotiation.cc:240] Negotiation successful
0929 10:59:18.225744 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":76,"thread_start_us":159,"threads_started":1}
I20260929 10:59:18.225826 28991 raft_consensus.cc:2454] T c1c0efaa113e406283b95025e32fc01d P cc42405df36e42629f0e2fd1a75c2751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f3290ac12b64068a3316deb116f06f8 in term 0.
I20260929 10:59:18.226084 29063 leader_election.cc:302] T c1c0efaa113e406283b95025e32fc01d P 9f3290ac12b64068a3316deb116f06f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f3290ac12b64068a3316deb116f06f8, cc42405df36e42629f0e2fd1a75c2751; no voters:
I20260929 10:59:18.226116 29273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1c0efaa113e406283b95025e32fc01d" candidate_uuid: "9f3290ac12b64068a3316deb116f06f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" is_pre_election: true
I20260929 10:59:18.226215 29348 raft_consensus.cc:2781] T c1c0efaa113e406283b95025e32fc01d P 9f3290ac12b64068a3316deb116f06f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:59:18.226235 29273 raft_consensus.cc:2454] T c1c0efaa113e406283b95025e32fc01d P 2bffd7b80ee6450fa765d1ed59a558c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f3290ac12b64068a3316deb116f06f8 in term 0.
I20260929 10:59:18.226254 29348 raft_consensus.cc:482] T c1c0efaa113e406283b95025e32fc01d P 9f3290ac12b64068a3316deb116f06f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:59:18.226275 29348 raft_consensus.cc:3037] T c1c0efaa113e406283b95025e32fc01d P 9f3290ac12b64068a3316deb116f06f8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:18.227259 29348 raft_consensus.cc:504] T c1c0efaa113e406283b95025e32fc01d P 9f3290ac12b64068a3316deb116f06f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } } peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } }
I20260929 10:59:18.227399 29348 leader_election.cc:288] T c1c0efaa113e406283b95025e32fc01d P 9f3290ac12b64068a3316deb116f06f8 [CANDIDATE]: Term 1 election: Requested vote from peers 2bffd7b80ee6450fa765d1ed59a558c0 (127.26.120.195:44043), cc42405df36e42629f0e2fd1a75c2751 (127.26.120.193:34187)
I20260929 10:59:18.227578 29273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1c0efaa113e406283b95025e32fc01d" candidate_uuid: "9f3290ac12b64068a3316deb116f06f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bffd7b80ee6450fa765d1ed59a558c0"
I20260929 10:59:18.227571 28991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1c0efaa113e406283b95025e32fc01d" candidate_uuid: "9f3290ac12b64068a3316deb116f06f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc42405df36e42629f0e2fd1a75c2751"
I20260929 10:59:18.227636 28991 raft_consensus.cc:3037] T c1c0efaa113e406283b95025e32fc01d P cc42405df36e42629f0e2fd1a75c2751 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:18.227638 29273 raft_consensus.cc:3037] T c1c0efaa113e406283b95025e32fc01d P 2bffd7b80ee6450fa765d1ed59a558c0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:18.228375 28930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:18.225033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:44043 (local address 127.26.120.193:46423)
0929 10:59:18.225118 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:18.225129 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:18.225168 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:18.225423 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:18.225424 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:18.225425 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:18.225478 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:18.225479 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.226299 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.226300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:18.227415 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:18.227417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.228153 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.228154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:18.228232 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.228235 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:59:18.228279 (+ 44us) client_negotiation.cc:240] Negotiation successful
0929 10:59:18.228311 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260929 10:59:18.228539 29273 raft_consensus.cc:2454] T c1c0efaa113e406283b95025e32fc01d P 2bffd7b80ee6450fa765d1ed59a558c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f3290ac12b64068a3316deb116f06f8 in term 1.
I20260929 10:59:18.228631 29338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:18.225273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:46423 (local address 127.26.120.195:44043)
0929 10:59:18.225303 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:18.225304 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:18.225311 (+ 7us) server_negotiation.cc:407] Connection header received
0929 10:59:18.225328 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:18.225329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:18.225337 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:18.225387 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:18.225555 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.226224 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.227503 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.228064 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.228120 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.228126 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:18.228526 (+ 400us) server_negotiation.cc:299] Negotiation successful
0929 10:59:18.228561 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16,"thread_start_us":79,"threads_started":1}
I20260929 10:59:18.228739 29064 leader_election.cc:302] T c1c0efaa113e406283b95025e32fc01d P 9f3290ac12b64068a3316deb116f06f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bffd7b80ee6450fa765d1ed59a558c0, 9f3290ac12b64068a3316deb116f06f8; no voters:
I20260929 10:59:18.228770 29273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1c0efaa113e406283b95025e32fc01d" candidate_uuid: "cc42405df36e42629f0e2fd1a75c2751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" is_pre_election: true
I20260929 10:59:18.228830 29348 raft_consensus.cc:2781] T c1c0efaa113e406283b95025e32fc01d P 9f3290ac12b64068a3316deb116f06f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:59:18.228878 29273 raft_consensus.cc:2379] T c1c0efaa113e406283b95025e32fc01d P 2bffd7b80ee6450fa765d1ed59a558c0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cc42405df36e42629f0e2fd1a75c2751 in current term 1: Already voted for candidate 9f3290ac12b64068a3316deb116f06f8 in this term.
I20260929 10:59:18.228883 28991 raft_consensus.cc:2454] T c1c0efaa113e406283b95025e32fc01d P cc42405df36e42629f0e2fd1a75c2751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f3290ac12b64068a3316deb116f06f8 in term 1.
I20260929 10:59:18.228950 29339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:18.225416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:52817 (local address 127.26.120.194:41721)
0929 10:59:18.225453 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:18.225455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:18.225579 (+ 124us) server_negotiation.cc:407] Connection header received
0929 10:59:18.225687 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:18.225688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:18.225698 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:18.225749 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:18.226140 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.226803 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.228053 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.228585 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.228624 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.228630 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:18.228863 (+ 233us) server_negotiation.cc:299] Negotiation successful
0929 10:59:18.228901 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 10:59:18.228979 29348 raft_consensus.cc:686] T c1c0efaa113e406283b95025e32fc01d P 9f3290ac12b64068a3316deb116f06f8 [term 1 LEADER]: Becoming Leader. State: Replica: 9f3290ac12b64068a3316deb116f06f8, State: Running, Role: LEADER
I20260929 10:59:18.229125 29350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:18.225098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:41721 (local address 127.26.120.193:52817)
0929 10:59:18.225513 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:18.225522 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:18.225605 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:18.225787 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:18.225788 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:18.225790 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:18.226033 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:18.226035 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.226899 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.226900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:18.227963 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:18.227965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.228698 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.228700 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:18.228771 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.228775 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:59:18.229013 (+ 238us) client_negotiation.cc:240] Negotiation successful
0929 10:59:18.229058 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":79,"threads_started":1}
I20260929 10:59:18.229092 29348 consensus_queue.cc:237] T c1c0efaa113e406283b95025e32fc01d P 9f3290ac12b64068a3316deb116f06f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } } peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } }
I20260929 10:59:18.229514 29131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1c0efaa113e406283b95025e32fc01d" candidate_uuid: "cc42405df36e42629f0e2fd1a75c2751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f3290ac12b64068a3316deb116f06f8" is_pre_election: true
I20260929 10:59:18.229857 28925 leader_election.cc:302] T c1c0efaa113e406283b95025e32fc01d P cc42405df36e42629f0e2fd1a75c2751 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc42405df36e42629f0e2fd1a75c2751; no voters: 2bffd7b80ee6450fa765d1ed59a558c0, 9f3290ac12b64068a3316deb116f06f8
I20260929 10:59:18.230011 29346 raft_consensus.cc:2726] T c1c0efaa113e406283b95025e32fc01d P cc42405df36e42629f0e2fd1a75c2751 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260929 10:59:18.229959 28834 catalog_manager.cc:5495] T c1c0efaa113e406283b95025e32fc01d P 9f3290ac12b64068a3316deb116f06f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f3290ac12b64068a3316deb116f06f8 (127.26.120.194). New cstate: current_term: 1 leader_uuid: "9f3290ac12b64068a3316deb116f06f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } health_report { overall_health: HEALTHY } } }
W20260929 10:59:18.256729 29178 tablet.cc:2369] T c1c0efaa113e406283b95025e32fc01d P 9f3290ac12b64068a3316deb116f06f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 29 10:59:18 dist-test-slave-2lzq krb5kdc[28789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679557, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:18.273244 29359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:18.266803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:42121 (local address 127.0.0.1:47790)
0929 10:59:18.266968 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:18.266980 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:18.267070 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:18.267197 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:18.267198 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:18.267489 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:18.267547 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:18.267549 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.268183 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.268184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:18.268699 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:18.268701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.269013 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.269013 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:18.269065 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.269569 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:18.269573 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:18.270904 (+ 1331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:18.272281 (+ 1377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:18.272288 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:18.272290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:18.272476 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:18.272668 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:18.272669 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:18.272669 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:18.272724 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:18.272947 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:18.272953 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:18.272997 (+ 44us) client_negotiation.cc:769] Sending connection context
0929 10:59:18.273057 (+ 60us) client_negotiation.cc:240] Negotiation successful
0929 10:59:18.273147 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":80,"threads_started":1}
I20260929 10:59:18.273355 28883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:18.266874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47790 (local address 127.26.120.254:42121)
0929 10:59:18.266990 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:18.266992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:18.267074 (+ 82us) server_negotiation.cc:407] Connection header received
0929 10:59:18.267117 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:18.267117 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:18.267126 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:18.267179 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:18.267627 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.268114 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.268772 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.268949 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.269001 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.270999 (+ 1998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:18.271000 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:18.271000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:18.271006 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:18.272203 (+ 1197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:18.272550 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:18.272551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:18.272552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:18.272606 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:18.272779 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:18.272779 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:18.272780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:18.272888 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:18.272939 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:18.273090 (+ 151us) server_negotiation.cc:299] Negotiation successful
0929 10:59:18.273192 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":85}
I20260929 10:59:18.275444 28834 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:47790:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 10:59:18.275619 28834 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:59:18.276980 28971 tablet_service.cc:1467] Processing CreateTablet for tablet a4c230b66fae45d78685be8735ed0b9a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=21ff7828d1f24467ab5a369a08ff07ca]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 10:59:18.276983 29111 tablet_service.cc:1467] Processing CreateTablet for tablet a4c230b66fae45d78685be8735ed0b9a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=21ff7828d1f24467ab5a369a08ff07ca]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 10:59:18.277153 28971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4c230b66fae45d78685be8735ed0b9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:18.277153 29111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4c230b66fae45d78685be8735ed0b9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:18.278294 29252 tablet_service.cc:1467] Processing CreateTablet for tablet a4c230b66fae45d78685be8735ed0b9a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=21ff7828d1f24467ab5a369a08ff07ca]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 10:59:18.278452 29252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4c230b66fae45d78685be8735ed0b9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:18.278718 29340 tablet_bootstrap.cc:492] T a4c230b66fae45d78685be8735ed0b9a P cc42405df36e42629f0e2fd1a75c2751: Bootstrap starting.
I20260929 10:59:18.278715 29341 tablet_bootstrap.cc:492] T a4c230b66fae45d78685be8735ed0b9a P 9f3290ac12b64068a3316deb116f06f8: Bootstrap starting.
I20260929 10:59:18.279130 29340 tablet_bootstrap.cc:654] T a4c230b66fae45d78685be8735ed0b9a P cc42405df36e42629f0e2fd1a75c2751: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:18.279140 29341 tablet_bootstrap.cc:654] T a4c230b66fae45d78685be8735ed0b9a P 9f3290ac12b64068a3316deb116f06f8: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:18.279807 29341 tablet_bootstrap.cc:492] T a4c230b66fae45d78685be8735ed0b9a P 9f3290ac12b64068a3316deb116f06f8: No bootstrap required, opened a new log
I20260929 10:59:18.279868 29341 ts_tablet_manager.cc:1397] T a4c230b66fae45d78685be8735ed0b9a P 9f3290ac12b64068a3316deb116f06f8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:59:18.280041 29342 tablet_bootstrap.cc:492] T a4c230b66fae45d78685be8735ed0b9a P 2bffd7b80ee6450fa765d1ed59a558c0: Bootstrap starting.
I20260929 10:59:18.280011 29341 raft_consensus.cc:348] T a4c230b66fae45d78685be8735ed0b9a P 9f3290ac12b64068a3316deb116f06f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } }
I20260929 10:59:18.280089 29341 raft_consensus.cc:374] T a4c230b66fae45d78685be8735ed0b9a P 9f3290ac12b64068a3316deb116f06f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:18.280115 29341 raft_consensus.cc:729] T a4c230b66fae45d78685be8735ed0b9a P 9f3290ac12b64068a3316deb116f06f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f3290ac12b64068a3316deb116f06f8, State: Initialized, Role: FOLLOWER
I20260929 10:59:18.280162 29341 consensus_queue.cc:260] T a4c230b66fae45d78685be8735ed0b9a P 9f3290ac12b64068a3316deb116f06f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } }
I20260929 10:59:18.280257 29341 ts_tablet_manager.cc:1428] T a4c230b66fae45d78685be8735ed0b9a P 9f3290ac12b64068a3316deb116f06f8: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260929 10:59:18.280366 29348 tablet_replica.cc:439] T a4c230b66fae45d78685be8735ed0b9a P 9f3290ac12b64068a3316deb116f06f8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } } }
I20260929 10:59:18.280426 29348 tablet_replica.cc:442] T a4c230b66fae45d78685be8735ed0b9a P 9f3290ac12b64068a3316deb116f06f8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:59:18.280726 29342 tablet_bootstrap.cc:654] T a4c230b66fae45d78685be8735ed0b9a P 2bffd7b80ee6450fa765d1ed59a558c0: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:18.280774 29340 tablet_bootstrap.cc:492] T a4c230b66fae45d78685be8735ed0b9a P cc42405df36e42629f0e2fd1a75c2751: No bootstrap required, opened a new log
I20260929 10:59:18.280836 29340 ts_tablet_manager.cc:1397] T a4c230b66fae45d78685be8735ed0b9a P cc42405df36e42629f0e2fd1a75c2751: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:59:18.280970 29340 raft_consensus.cc:348] T a4c230b66fae45d78685be8735ed0b9a P cc42405df36e42629f0e2fd1a75c2751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } }
I20260929 10:59:18.281016 29340 raft_consensus.cc:374] T a4c230b66fae45d78685be8735ed0b9a P cc42405df36e42629f0e2fd1a75c2751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:18.281032 29340 raft_consensus.cc:729] T a4c230b66fae45d78685be8735ed0b9a P cc42405df36e42629f0e2fd1a75c2751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc42405df36e42629f0e2fd1a75c2751, State: Initialized, Role: FOLLOWER
I20260929 10:59:18.281088 29340 consensus_queue.cc:260] T a4c230b66fae45d78685be8735ed0b9a P cc42405df36e42629f0e2fd1a75c2751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } }
I20260929 10:59:18.281173 29340 ts_tablet_manager.cc:1428] T a4c230b66fae45d78685be8735ed0b9a P cc42405df36e42629f0e2fd1a75c2751: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:18.281224 29346 tablet_replica.cc:439] T a4c230b66fae45d78685be8735ed0b9a P cc42405df36e42629f0e2fd1a75c2751: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } } }
I20260929 10:59:18.281301 29346 tablet_replica.cc:442] T a4c230b66fae45d78685be8735ed0b9a P cc42405df36e42629f0e2fd1a75c2751: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:59:18.281394 29342 tablet_bootstrap.cc:492] T a4c230b66fae45d78685be8735ed0b9a P 2bffd7b80ee6450fa765d1ed59a558c0: No bootstrap required, opened a new log
I20260929 10:59:18.281443 29342 ts_tablet_manager.cc:1397] T a4c230b66fae45d78685be8735ed0b9a P 2bffd7b80ee6450fa765d1ed59a558c0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.001s
I20260929 10:59:18.281615 29342 raft_consensus.cc:348] T a4c230b66fae45d78685be8735ed0b9a P 2bffd7b80ee6450fa765d1ed59a558c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } }
I20260929 10:59:18.281690 29342 raft_consensus.cc:374] T a4c230b66fae45d78685be8735ed0b9a P 2bffd7b80ee6450fa765d1ed59a558c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:18.281713 29342 raft_consensus.cc:729] T a4c230b66fae45d78685be8735ed0b9a P 2bffd7b80ee6450fa765d1ed59a558c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bffd7b80ee6450fa765d1ed59a558c0, State: Initialized, Role: FOLLOWER
I20260929 10:59:18.281754 29342 consensus_queue.cc:260] T a4c230b66fae45d78685be8735ed0b9a P 2bffd7b80ee6450fa765d1ed59a558c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } }
I20260929 10:59:18.281855 29342 ts_tablet_manager.cc:1428] T a4c230b66fae45d78685be8735ed0b9a P 2bffd7b80ee6450fa765d1ed59a558c0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:18.281983 29347 tablet_replica.cc:439] T a4c230b66fae45d78685be8735ed0b9a P 2bffd7b80ee6450fa765d1ed59a558c0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } } }
I20260929 10:59:18.282084 29347 tablet_replica.cc:442] T a4c230b66fae45d78685be8735ed0b9a P 2bffd7b80ee6450fa765d1ed59a558c0: This TxnStatusTablet replica's current role is: FOLLOWER
W20260929 10:59:18.352186 29038 tablet.cc:2369] T a4c230b66fae45d78685be8735ed0b9a P cc42405df36e42629f0e2fd1a75c2751: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:59:18.402971 29320 tablet.cc:2369] T a4c230b66fae45d78685be8735ed0b9a P 2bffd7b80ee6450fa765d1ed59a558c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:59:18.438465 29348 raft_consensus.cc:482] T a4c230b66fae45d78685be8735ed0b9a P 9f3290ac12b64068a3316deb116f06f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:59:18.438560 29348 raft_consensus.cc:504] T a4c230b66fae45d78685be8735ed0b9a P 9f3290ac12b64068a3316deb116f06f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } }
I20260929 10:59:18.438741 29348 leader_election.cc:288] T a4c230b66fae45d78685be8735ed0b9a P 9f3290ac12b64068a3316deb116f06f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc42405df36e42629f0e2fd1a75c2751 (127.26.120.193:34187), 2bffd7b80ee6450fa765d1ed59a558c0 (127.26.120.195:44043)
I20260929 10:59:18.438982 29273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4c230b66fae45d78685be8735ed0b9a" candidate_uuid: "9f3290ac12b64068a3316deb116f06f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" is_pre_election: true
I20260929 10:59:18.438994 28991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4c230b66fae45d78685be8735ed0b9a" candidate_uuid: "9f3290ac12b64068a3316deb116f06f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc42405df36e42629f0e2fd1a75c2751" is_pre_election: true
I20260929 10:59:18.439093 29273 raft_consensus.cc:2454] T a4c230b66fae45d78685be8735ed0b9a P 2bffd7b80ee6450fa765d1ed59a558c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f3290ac12b64068a3316deb116f06f8 in term 0.
I20260929 10:59:18.439090 28991 raft_consensus.cc:2454] T a4c230b66fae45d78685be8735ed0b9a P cc42405df36e42629f0e2fd1a75c2751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f3290ac12b64068a3316deb116f06f8 in term 0.
I20260929 10:59:18.439252 29064 leader_election.cc:302] T a4c230b66fae45d78685be8735ed0b9a P 9f3290ac12b64068a3316deb116f06f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bffd7b80ee6450fa765d1ed59a558c0, 9f3290ac12b64068a3316deb116f06f8; no voters:
I20260929 10:59:18.439360 29348 raft_consensus.cc:2781] T a4c230b66fae45d78685be8735ed0b9a P 9f3290ac12b64068a3316deb116f06f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:59:18.439407 29348 raft_consensus.cc:482] T a4c230b66fae45d78685be8735ed0b9a P 9f3290ac12b64068a3316deb116f06f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:59:18.439421 29348 raft_consensus.cc:3037] T a4c230b66fae45d78685be8735ed0b9a P 9f3290ac12b64068a3316deb116f06f8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:18.440080 29348 raft_consensus.cc:504] T a4c230b66fae45d78685be8735ed0b9a P 9f3290ac12b64068a3316deb116f06f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } }
I20260929 10:59:18.440187 29348 leader_election.cc:288] T a4c230b66fae45d78685be8735ed0b9a P 9f3290ac12b64068a3316deb116f06f8 [CANDIDATE]: Term 1 election: Requested vote from peers cc42405df36e42629f0e2fd1a75c2751 (127.26.120.193:34187), 2bffd7b80ee6450fa765d1ed59a558c0 (127.26.120.195:44043)
I20260929 10:59:18.440402 28991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4c230b66fae45d78685be8735ed0b9a" candidate_uuid: "9f3290ac12b64068a3316deb116f06f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc42405df36e42629f0e2fd1a75c2751"
I20260929 10:59:18.440462 28991 raft_consensus.cc:3037] T a4c230b66fae45d78685be8735ed0b9a P cc42405df36e42629f0e2fd1a75c2751 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:18.441058 28991 raft_consensus.cc:2454] T a4c230b66fae45d78685be8735ed0b9a P cc42405df36e42629f0e2fd1a75c2751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f3290ac12b64068a3316deb116f06f8 in term 1.
I20260929 10:59:18.441174 29273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4c230b66fae45d78685be8735ed0b9a" candidate_uuid: "9f3290ac12b64068a3316deb116f06f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bffd7b80ee6450fa765d1ed59a558c0"
I20260929 10:59:18.441208 29063 leader_election.cc:302] T a4c230b66fae45d78685be8735ed0b9a P 9f3290ac12b64068a3316deb116f06f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f3290ac12b64068a3316deb116f06f8, cc42405df36e42629f0e2fd1a75c2751; no voters:
I20260929 10:59:18.441249 29273 raft_consensus.cc:3037] T a4c230b66fae45d78685be8735ed0b9a P 2bffd7b80ee6450fa765d1ed59a558c0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:18.441300 29348 raft_consensus.cc:2781] T a4c230b66fae45d78685be8735ed0b9a P 9f3290ac12b64068a3316deb116f06f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:59:18.441361 29348 raft_consensus.cc:686] T a4c230b66fae45d78685be8735ed0b9a P 9f3290ac12b64068a3316deb116f06f8 [term 1 LEADER]: Becoming Leader. State: Replica: 9f3290ac12b64068a3316deb116f06f8, State: Running, Role: LEADER
I20260929 10:59:18.441417 29348 consensus_queue.cc:237] T a4c230b66fae45d78685be8735ed0b9a P 9f3290ac12b64068a3316deb116f06f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } }
I20260929 10:59:18.441704 29352 tablet_replica.cc:439] T a4c230b66fae45d78685be8735ed0b9a P 9f3290ac12b64068a3316deb116f06f8: TxnStatusTablet state changed. Reason: New leader 9f3290ac12b64068a3316deb116f06f8. Latest consensus state: current_term: 1 leader_uuid: "9f3290ac12b64068a3316deb116f06f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } } }
I20260929 10:59:18.441792 29352 tablet_replica.cc:442] T a4c230b66fae45d78685be8735ed0b9a P 9f3290ac12b64068a3316deb116f06f8: This TxnStatusTablet replica's current role is: LEADER
I20260929 10:59:18.442026 29364 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 10:59:18.442123 29273 raft_consensus.cc:2454] T a4c230b66fae45d78685be8735ed0b9a P 2bffd7b80ee6450fa765d1ed59a558c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f3290ac12b64068a3316deb116f06f8 in term 1.
I20260929 10:59:18.442260 29364 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 10:59:18.442297 28834 catalog_manager.cc:5495] T a4c230b66fae45d78685be8735ed0b9a P 9f3290ac12b64068a3316deb116f06f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f3290ac12b64068a3316deb116f06f8 (127.26.120.194). New cstate: current_term: 1 leader_uuid: "9f3290ac12b64068a3316deb116f06f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } health_report { overall_health: UNKNOWN } } }
I20260929 10:59:18.464416 29359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:18.461774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:41721 (local address 127.0.0.1:46934)
0929 10:59:18.461873 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:18.461900 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:18.461954 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:18.462176 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:18.462177 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:18.462178 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 10:59:18.462229 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:18.462231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.462919 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.462920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:18.463499 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:18.463501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.463885 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.463886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:18.463963 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.463977 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 10:59:18.464238 (+ 261us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 10:59:18.464247 (+ 9us) client_negotiation.cc:769] Sending connection context
0929 10:59:18.464296 (+ 49us) client_negotiation.cc:240] Negotiation successful
0929 10:59:18.464350 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260929 10:59:18.464412 29339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:18.461864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46934 (local address 127.26.120.194:41721)
0929 10:59:18.461928 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:18.461930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:18.461939 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:59:18.462070 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:18.462071 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:18.462090 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:18.462170 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:59:18.462306 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.462818 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.463597 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.463793 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.463882 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.464070 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:59:18.464171 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:59:18.464261 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:18.464305 (+ 44us) server_negotiation.cc:299] Negotiation successful
0929 10:59:18.464337 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 10:59:18.466346 29273 raft_consensus.cc:1261] T a4c230b66fae45d78685be8735ed0b9a P 2bffd7b80ee6450fa765d1ed59a558c0 [term 1 FOLLOWER]: Refusing update from remote peer 9f3290ac12b64068a3316deb116f06f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:59:18.466374 28991 raft_consensus.cc:1261] T a4c230b66fae45d78685be8735ed0b9a P cc42405df36e42629f0e2fd1a75c2751 [term 1 FOLLOWER]: Refusing update from remote peer 9f3290ac12b64068a3316deb116f06f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:59:18.466629 29352 consensus_queue.cc:1035] T a4c230b66fae45d78685be8735ed0b9a P 9f3290ac12b64068a3316deb116f06f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:59:18.466742 29348 consensus_queue.cc:1035] T a4c230b66fae45d78685be8735ed0b9a P 9f3290ac12b64068a3316deb116f06f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:59:18.467865 29346 tablet_replica.cc:439] T a4c230b66fae45d78685be8735ed0b9a P cc42405df36e42629f0e2fd1a75c2751: TxnStatusTablet state changed. Reason: New leader 9f3290ac12b64068a3316deb116f06f8. Latest consensus state: current_term: 1 leader_uuid: "9f3290ac12b64068a3316deb116f06f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } } }
I20260929 10:59:18.467967 29346 tablet_replica.cc:442] T a4c230b66fae45d78685be8735ed0b9a P cc42405df36e42629f0e2fd1a75c2751: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:59:18.467988 29347 tablet_replica.cc:439] T a4c230b66fae45d78685be8735ed0b9a P 2bffd7b80ee6450fa765d1ed59a558c0: TxnStatusTablet state changed. Reason: New leader 9f3290ac12b64068a3316deb116f06f8. Latest consensus state: current_term: 1 leader_uuid: "9f3290ac12b64068a3316deb116f06f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } } }
I20260929 10:59:18.468106 29347 tablet_replica.cc:442] T a4c230b66fae45d78685be8735ed0b9a P 2bffd7b80ee6450fa765d1ed59a558c0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:59:18.468947 29348 tablet_replica.cc:439] T a4c230b66fae45d78685be8735ed0b9a P 9f3290ac12b64068a3316deb116f06f8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9f3290ac12b64068a3316deb116f06f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } } }
I20260929 10:59:18.469031 29348 tablet_replica.cc:442] T a4c230b66fae45d78685be8735ed0b9a P 9f3290ac12b64068a3316deb116f06f8: This TxnStatusTablet replica's current role is: LEADER
I20260929 10:59:18.469017 29347 tablet_replica.cc:439] T a4c230b66fae45d78685be8735ed0b9a P 2bffd7b80ee6450fa765d1ed59a558c0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9f3290ac12b64068a3316deb116f06f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } } }
I20260929 10:59:18.469103 29347 tablet_replica.cc:442] T a4c230b66fae45d78685be8735ed0b9a P 2bffd7b80ee6450fa765d1ed59a558c0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:59:18.469097 29367 mvcc.cc:205] Tried to move back new op lower bound from 7334623471475470336 to 7334623471376244736. Current Snapshot: MvccSnapshot[applied={T|T < 7334623471475470336}]
I20260929 10:59:18.469991 29352 tablet_replica.cc:439] T a4c230b66fae45d78685be8735ed0b9a P 9f3290ac12b64068a3316deb116f06f8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9f3290ac12b64068a3316deb116f06f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } } }
I20260929 10:59:18.470104 29352 tablet_replica.cc:442] T a4c230b66fae45d78685be8735ed0b9a P 9f3290ac12b64068a3316deb116f06f8: This TxnStatusTablet replica's current role is: LEADER
I20260929 10:59:18.470420 29368 mvcc.cc:205] Tried to move back new op lower bound from 7334623471475470336 to 7334623471376244736. Current Snapshot: MvccSnapshot[applied={T|T < 7334623471475470336}]
I20260929 10:59:18.470518 29346 tablet_replica.cc:439] T a4c230b66fae45d78685be8735ed0b9a P cc42405df36e42629f0e2fd1a75c2751: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9f3290ac12b64068a3316deb116f06f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } } }
I20260929 10:59:18.470597 29346 tablet_replica.cc:442] T a4c230b66fae45d78685be8735ed0b9a P cc42405df36e42629f0e2fd1a75c2751: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:59:18.471652 29369 mvcc.cc:205] Tried to move back new op lower bound from 7334623471475470336 to 7334623471376244736. Current Snapshot: MvccSnapshot[applied={T|T < 7334623471475470336}]
I20260929 10:59:18.473735 29339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:18.471336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46942 (local address 127.26.120.194:41721)
0929 10:59:18.471375 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:18.471377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:18.471385 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:59:18.471407 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:18.471407 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:18.471417 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:18.471471 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:59:18.471619 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.472230 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.473029 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.473228 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.473299 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.473477 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:59:18.473517 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:59:18.473579 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:18.473643 (+ 64us) server_negotiation.cc:299] Negotiation successful
0929 10:59:18.473662 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 10:59:18.478997 29339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:18.476374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:53961 (local address 127.26.120.194:41721)
0929 10:59:18.476428 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:18.476430 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:18.476440 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:59:18.476462 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:18.476463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:18.476472 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:18.476540 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:59:18.476631 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.477348 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.478192 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.478489 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.478559 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.478750 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:59:18.478781 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:59:18.478835 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:18.478880 (+ 45us) server_negotiation.cc:299] Negotiation successful
0929 10:59:18.478898 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260929 10:59:18.478999 29185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:18.476293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:41721 (local address 127.26.120.194:53961)
0929 10:59:18.476373 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:18.476383 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:18.476422 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:18.476540 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:18.476541 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:18.476542 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 10:59:18.476592 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:18.476593 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.477463 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.477464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:18.478115 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:18.478118 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.478567 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.478568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:18.478641 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.478660 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 10:59:18.478841 (+ 181us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 10:59:18.478842 (+ 1us) client_negotiation.cc:769] Sending connection context
0929 10:59:18.478877 (+ 35us) client_negotiation.cc:240] Negotiation successful
0929 10:59:18.478920 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260929 10:59:18.481304 29273 raft_consensus.cc:1261] T c1c0efaa113e406283b95025e32fc01d P 2bffd7b80ee6450fa765d1ed59a558c0 [term 1 FOLLOWER]: Refusing update from remote peer 9f3290ac12b64068a3316deb116f06f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:59:18.481349 28991 raft_consensus.cc:1261] T c1c0efaa113e406283b95025e32fc01d P cc42405df36e42629f0e2fd1a75c2751 [term 1 FOLLOWER]: Refusing update from remote peer 9f3290ac12b64068a3316deb116f06f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:59:18.481503 29373 consensus_queue.cc:1035] T c1c0efaa113e406283b95025e32fc01d P 9f3290ac12b64068a3316deb116f06f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:59:18.481608 29352 consensus_queue.cc:1035] T c1c0efaa113e406283b95025e32fc01d P 9f3290ac12b64068a3316deb116f06f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:59:18.492221 28834 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:59:18.492328 28834 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:59:18.493062 28834 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9ffa6da9b319473ea32390a2c8410b30: Sending DeleteTablet for 3 replicas of tablet c1c0efaa113e406283b95025e32fc01d
I20260929 10:59:18.493353 28971 tablet_service.cc:1514] Processing DeleteTablet for tablet c1c0efaa113e406283b95025e32fc01d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:18 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:60498
I20260929 10:59:18.493433 29252 tablet_service.cc:1514] Processing DeleteTablet for tablet c1c0efaa113e406283b95025e32fc01d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:18 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:45592
I20260929 10:59:18.493659 29383 tablet_replica.cc:330] T c1c0efaa113e406283b95025e32fc01d P 2bffd7b80ee6450fa765d1ed59a558c0: stopping tablet replica
I20260929 10:59:18.493683 29110 tablet_service.cc:1514] Processing DeleteTablet for tablet c1c0efaa113e406283b95025e32fc01d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:18 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:46918
I20260929 10:59:18.493767 29383 raft_consensus.cc:2229] T c1c0efaa113e406283b95025e32fc01d P 2bffd7b80ee6450fa765d1ed59a558c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:59:18.493829 29383 raft_consensus.cc:2258] T c1c0efaa113e406283b95025e32fc01d P 2bffd7b80ee6450fa765d1ed59a558c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:18.493865 28834 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47786:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:59:18.493994 28834 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:59:18.494256 29383 ts_tablet_manager.cc:1905] T c1c0efaa113e406283b95025e32fc01d P 2bffd7b80ee6450fa765d1ed59a558c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:18.495090 28971 tablet_service.cc:1467] Processing CreateTablet for tablet af75ff380e584b218fd02c8afaf0485d (DEFAULT_TABLE table=test-table [id=b467eaab7c3e48488065bfc3440f9d59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:18.495177 29252 tablet_service.cc:1467] Processing CreateTablet for tablet af75ff380e584b218fd02c8afaf0485d (DEFAULT_TABLE table=test-table [id=b467eaab7c3e48488065bfc3440f9d59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:18.495206 28971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af75ff380e584b218fd02c8afaf0485d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:18.495287 29252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af75ff380e584b218fd02c8afaf0485d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:18.496264 29340 tablet_bootstrap.cc:492] T af75ff380e584b218fd02c8afaf0485d P cc42405df36e42629f0e2fd1a75c2751: Bootstrap starting.
I20260929 10:59:18.496467 29342 tablet_bootstrap.cc:492] T af75ff380e584b218fd02c8afaf0485d P 2bffd7b80ee6450fa765d1ed59a558c0: Bootstrap starting.
I20260929 10:59:18.496649 29340 tablet_bootstrap.cc:654] T af75ff380e584b218fd02c8afaf0485d P cc42405df36e42629f0e2fd1a75c2751: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:18.496832 29342 tablet_bootstrap.cc:654] T af75ff380e584b218fd02c8afaf0485d P 2bffd7b80ee6450fa765d1ed59a558c0: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:18.494287 29384 tablet_replica.cc:330] T c1c0efaa113e406283b95025e32fc01d P 9f3290ac12b64068a3316deb116f06f8: stopping tablet replica
I20260929 10:59:18.495107 29110 tablet_service.cc:1467] Processing CreateTablet for tablet af75ff380e584b218fd02c8afaf0485d (DEFAULT_TABLE table=test-table [id=b467eaab7c3e48488065bfc3440f9d59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:18.497259 29110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af75ff380e584b218fd02c8afaf0485d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:18.497520 29383 ts_tablet_manager.cc:1918] T c1c0efaa113e406283b95025e32fc01d P 2bffd7b80ee6450fa765d1ed59a558c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:59:18.497604 29383 log.cc:1196] T c1c0efaa113e406283b95025e32fc01d P 2bffd7b80ee6450fa765d1ed59a558c0: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-2/wal/wals/c1c0efaa113e406283b95025e32fc01d
I20260929 10:59:18.497838 29383 ts_tablet_manager.cc:1939] T c1c0efaa113e406283b95025e32fc01d P 2bffd7b80ee6450fa765d1ed59a558c0: Deleting consensus metadata
I20260929 10:59:18.498442 29340 tablet_bootstrap.cc:492] T af75ff380e584b218fd02c8afaf0485d P cc42405df36e42629f0e2fd1a75c2751: No bootstrap required, opened a new log
I20260929 10:59:18.498502 29340 ts_tablet_manager.cc:1397] T af75ff380e584b218fd02c8afaf0485d P cc42405df36e42629f0e2fd1a75c2751: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:59:18.498651 29341 tablet_bootstrap.cc:492] T af75ff380e584b218fd02c8afaf0485d P 9f3290ac12b64068a3316deb116f06f8: Bootstrap starting.
I20260929 10:59:18.498647 29340 raft_consensus.cc:348] T af75ff380e584b218fd02c8afaf0485d P cc42405df36e42629f0e2fd1a75c2751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } }
I20260929 10:59:18.498711 29340 raft_consensus.cc:374] T af75ff380e584b218fd02c8afaf0485d P cc42405df36e42629f0e2fd1a75c2751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:18.498731 29340 raft_consensus.cc:729] T af75ff380e584b218fd02c8afaf0485d P cc42405df36e42629f0e2fd1a75c2751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc42405df36e42629f0e2fd1a75c2751, State: Initialized, Role: FOLLOWER
I20260929 10:59:18.498781 29340 consensus_queue.cc:260] T af75ff380e584b218fd02c8afaf0485d P cc42405df36e42629f0e2fd1a75c2751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } }
I20260929 10:59:18.498814 28820 catalog_manager.cc:4841] TS 2bffd7b80ee6450fa765d1ed59a558c0 (127.26.120.195:44043): tablet c1c0efaa113e406283b95025e32fc01d (table test-table [id=c59c0a8921c64c6cb8aa5f4fb3969edb]) successfully deleted
I20260929 10:59:18.498878 29340 ts_tablet_manager.cc:1428] T af75ff380e584b218fd02c8afaf0485d P cc42405df36e42629f0e2fd1a75c2751: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260929 10:59:18.498957 29384 raft_consensus.cc:2229] T c1c0efaa113e406283b95025e32fc01d P 9f3290ac12b64068a3316deb116f06f8 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:59:18.499069 29341 tablet_bootstrap.cc:654] T af75ff380e584b218fd02c8afaf0485d P 9f3290ac12b64068a3316deb116f06f8: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:18.499063 29384 raft_consensus.cc:2258] T c1c0efaa113e406283b95025e32fc01d P 9f3290ac12b64068a3316deb116f06f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:18.499794 29341 tablet_bootstrap.cc:492] T af75ff380e584b218fd02c8afaf0485d P 9f3290ac12b64068a3316deb116f06f8: No bootstrap required, opened a new log
I20260929 10:59:18.499840 29341 ts_tablet_manager.cc:1397] T af75ff380e584b218fd02c8afaf0485d P 9f3290ac12b64068a3316deb116f06f8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:59:18.499949 29341 raft_consensus.cc:348] T af75ff380e584b218fd02c8afaf0485d P 9f3290ac12b64068a3316deb116f06f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } }
I20260929 10:59:18.500005 29341 raft_consensus.cc:374] T af75ff380e584b218fd02c8afaf0485d P 9f3290ac12b64068a3316deb116f06f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:18.500022 29341 raft_consensus.cc:729] T af75ff380e584b218fd02c8afaf0485d P 9f3290ac12b64068a3316deb116f06f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f3290ac12b64068a3316deb116f06f8, State: Initialized, Role: FOLLOWER
I20260929 10:59:18.500070 29341 consensus_queue.cc:260] T af75ff380e584b218fd02c8afaf0485d P 9f3290ac12b64068a3316deb116f06f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } }
I20260929 10:59:18.500159 29341 ts_tablet_manager.cc:1428] T af75ff380e584b218fd02c8afaf0485d P 9f3290ac12b64068a3316deb116f06f8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:18.500425 29384 ts_tablet_manager.cc:1905] T c1c0efaa113e406283b95025e32fc01d P 9f3290ac12b64068a3316deb116f06f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:18.501823 29384 ts_tablet_manager.cc:1918] T c1c0efaa113e406283b95025e32fc01d P 9f3290ac12b64068a3316deb116f06f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:59:18.501891 29384 log.cc:1196] T c1c0efaa113e406283b95025e32fc01d P 9f3290ac12b64068a3316deb116f06f8: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-1/wal/wals/c1c0efaa113e406283b95025e32fc01d
I20260929 10:59:18.502193 29384 ts_tablet_manager.cc:1939] T c1c0efaa113e406283b95025e32fc01d P 9f3290ac12b64068a3316deb116f06f8: Deleting consensus metadata
I20260929 10:59:18.502343 29382 tablet_replica.cc:330] T c1c0efaa113e406283b95025e32fc01d P cc42405df36e42629f0e2fd1a75c2751: stopping tablet replica
I20260929 10:59:18.502416 29342 tablet_bootstrap.cc:492] T af75ff380e584b218fd02c8afaf0485d P 2bffd7b80ee6450fa765d1ed59a558c0: No bootstrap required, opened a new log
I20260929 10:59:18.502467 29342 ts_tablet_manager.cc:1397] T af75ff380e584b218fd02c8afaf0485d P 2bffd7b80ee6450fa765d1ed59a558c0: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.000s
I20260929 10:59:18.502477 29382 raft_consensus.cc:2229] T c1c0efaa113e406283b95025e32fc01d P cc42405df36e42629f0e2fd1a75c2751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:59:18.502534 29382 raft_consensus.cc:2258] T c1c0efaa113e406283b95025e32fc01d P cc42405df36e42629f0e2fd1a75c2751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:18.502604 29342 raft_consensus.cc:348] T af75ff380e584b218fd02c8afaf0485d P 2bffd7b80ee6450fa765d1ed59a558c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } }
I20260929 10:59:18.502668 29342 raft_consensus.cc:374] T af75ff380e584b218fd02c8afaf0485d P 2bffd7b80ee6450fa765d1ed59a558c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:18.502688 29342 raft_consensus.cc:729] T af75ff380e584b218fd02c8afaf0485d P 2bffd7b80ee6450fa765d1ed59a558c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bffd7b80ee6450fa765d1ed59a558c0, State: Initialized, Role: FOLLOWER
I20260929 10:59:18.502731 29342 consensus_queue.cc:260] T af75ff380e584b218fd02c8afaf0485d P 2bffd7b80ee6450fa765d1ed59a558c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } }
I20260929 10:59:18.502815 29342 ts_tablet_manager.cc:1428] T af75ff380e584b218fd02c8afaf0485d P 2bffd7b80ee6450fa765d1ed59a558c0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:18.502897 29382 ts_tablet_manager.cc:1905] T c1c0efaa113e406283b95025e32fc01d P cc42405df36e42629f0e2fd1a75c2751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:18.503350 28821 catalog_manager.cc:4841] TS 9f3290ac12b64068a3316deb116f06f8 (127.26.120.194:41721): tablet c1c0efaa113e406283b95025e32fc01d (table test-table [id=c59c0a8921c64c6cb8aa5f4fb3969edb]) successfully deleted
I20260929 10:59:18.504271 29382 ts_tablet_manager.cc:1918] T c1c0efaa113e406283b95025e32fc01d P cc42405df36e42629f0e2fd1a75c2751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:59:18.504360 29382 log.cc:1196] T c1c0efaa113e406283b95025e32fc01d P cc42405df36e42629f0e2fd1a75c2751: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-0/wal/wals/c1c0efaa113e406283b95025e32fc01d
I20260929 10:59:18.504664 29382 ts_tablet_manager.cc:1939] T c1c0efaa113e406283b95025e32fc01d P cc42405df36e42629f0e2fd1a75c2751: Deleting consensus metadata
I20260929 10:59:18.505062 28819 catalog_manager.cc:4841] TS cc42405df36e42629f0e2fd1a75c2751 (127.26.120.193:34187): tablet c1c0efaa113e406283b95025e32fc01d (table test-table [id=c59c0a8921c64c6cb8aa5f4fb3969edb]) successfully deleted
I20260929 10:59:18.669515 29347 raft_consensus.cc:482] T af75ff380e584b218fd02c8afaf0485d P 2bffd7b80ee6450fa765d1ed59a558c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:59:18.669644 29347 raft_consensus.cc:504] T af75ff380e584b218fd02c8afaf0485d P 2bffd7b80ee6450fa765d1ed59a558c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } }
I20260929 10:59:18.670027 29347 leader_election.cc:288] T af75ff380e584b218fd02c8afaf0485d P 2bffd7b80ee6450fa765d1ed59a558c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc42405df36e42629f0e2fd1a75c2751 (127.26.120.193:34187), 9f3290ac12b64068a3316deb116f06f8 (127.26.120.194:41721)
I20260929 10:59:18.673571 29339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:18.670381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:54795 (local address 127.26.120.194:41721)
0929 10:59:18.670436 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:18.670439 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:18.670493 (+ 54us) server_negotiation.cc:407] Connection header received
0929 10:59:18.670615 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:18.670616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:18.670629 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:18.670703 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:18.670894 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.671388 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.672660 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.673221 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.673299 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.673307 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:18.673472 (+ 165us) server_negotiation.cc:299] Negotiation successful
0929 10:59:18.673518 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 10:59:18.673774 29389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:18.670236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:34187 (local address 127.26.120.195:35171)
0929 10:59:18.670379 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:18.670391 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:18.670453 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:18.670697 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:18.670697 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:18.670700 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:18.670771 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:18.670774 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.671936 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.671936 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:18.672866 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:18.672868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.673459 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.673460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:18.673549 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.673552 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:59:18.673641 (+ 89us) client_negotiation.cc:240] Negotiation successful
0929 10:59:18.673687 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":67,"threads_started":1}
I20260929 10:59:18.673800 29337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:18.670492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:35171 (local address 127.26.120.193:34187)
0929 10:59:18.670553 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:18.670555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:18.670563 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:59:18.670592 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:18.670593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:18.670602 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:18.670689 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:18.670893 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.671831 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.672947 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.673367 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.673440 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.673450 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:18.673667 (+ 217us) server_negotiation.cc:299] Negotiation successful
0929 10:59:18.673720 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 10:59:18.674018 28991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af75ff380e584b218fd02c8afaf0485d" candidate_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc42405df36e42629f0e2fd1a75c2751" is_pre_election: true
I20260929 10:59:18.674135 28991 raft_consensus.cc:2454] T af75ff380e584b218fd02c8afaf0485d P cc42405df36e42629f0e2fd1a75c2751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2bffd7b80ee6450fa765d1ed59a558c0 in term 0.
I20260929 10:59:18.674197 29388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:18.670242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:41721 (local address 127.26.120.195:54795)
0929 10:59:18.670435 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:18.670444 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:18.670516 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:18.670754 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:18.670755 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:18.670758 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:18.670803 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:18.670805 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.671476 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.671477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:18.672566 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:18.672569 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.673306 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.673307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:18.673380 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.673385 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 10:59:18.673995 (+ 610us) client_negotiation.cc:240] Negotiation successful
0929 10:59:18.674081 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":53,"threads_started":1}
I20260929 10:59:18.674324 29204 leader_election.cc:302] T af75ff380e584b218fd02c8afaf0485d P 2bffd7b80ee6450fa765d1ed59a558c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bffd7b80ee6450fa765d1ed59a558c0, cc42405df36e42629f0e2fd1a75c2751; no voters:
I20260929 10:59:18.674445 29347 raft_consensus.cc:2781] T af75ff380e584b218fd02c8afaf0485d P 2bffd7b80ee6450fa765d1ed59a558c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:59:18.674476 29347 raft_consensus.cc:482] T af75ff380e584b218fd02c8afaf0485d P 2bffd7b80ee6450fa765d1ed59a558c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:59:18.674495 29347 raft_consensus.cc:3037] T af75ff380e584b218fd02c8afaf0485d P 2bffd7b80ee6450fa765d1ed59a558c0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:18.674445 29131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af75ff380e584b218fd02c8afaf0485d" candidate_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f3290ac12b64068a3316deb116f06f8" is_pre_election: true
I20260929 10:59:18.674531 29131 raft_consensus.cc:2454] T af75ff380e584b218fd02c8afaf0485d P 9f3290ac12b64068a3316deb116f06f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2bffd7b80ee6450fa765d1ed59a558c0 in term 0.
I20260929 10:59:18.675251 29347 raft_consensus.cc:504] T af75ff380e584b218fd02c8afaf0485d P 2bffd7b80ee6450fa765d1ed59a558c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } }
I20260929 10:59:18.675384 29347 leader_election.cc:288] T af75ff380e584b218fd02c8afaf0485d P 2bffd7b80ee6450fa765d1ed59a558c0 [CANDIDATE]: Term 1 election: Requested vote from peers cc42405df36e42629f0e2fd1a75c2751 (127.26.120.193:34187), 9f3290ac12b64068a3316deb116f06f8 (127.26.120.194:41721)
I20260929 10:59:18.675519 29131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af75ff380e584b218fd02c8afaf0485d" candidate_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f3290ac12b64068a3316deb116f06f8"
I20260929 10:59:18.675513 28991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af75ff380e584b218fd02c8afaf0485d" candidate_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc42405df36e42629f0e2fd1a75c2751"
I20260929 10:59:18.675590 29131 raft_consensus.cc:3037] T af75ff380e584b218fd02c8afaf0485d P 9f3290ac12b64068a3316deb116f06f8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:18.675591 28991 raft_consensus.cc:3037] T af75ff380e584b218fd02c8afaf0485d P cc42405df36e42629f0e2fd1a75c2751 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:18.676363 28991 raft_consensus.cc:2454] T af75ff380e584b218fd02c8afaf0485d P cc42405df36e42629f0e2fd1a75c2751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2bffd7b80ee6450fa765d1ed59a558c0 in term 1.
I20260929 10:59:18.676383 29131 raft_consensus.cc:2454] T af75ff380e584b218fd02c8afaf0485d P 9f3290ac12b64068a3316deb116f06f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2bffd7b80ee6450fa765d1ed59a558c0 in term 1.
I20260929 10:59:18.676532 29204 leader_election.cc:302] T af75ff380e584b218fd02c8afaf0485d P 2bffd7b80ee6450fa765d1ed59a558c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bffd7b80ee6450fa765d1ed59a558c0, cc42405df36e42629f0e2fd1a75c2751; no voters:
I20260929 10:59:18.676637 29347 raft_consensus.cc:2781] T af75ff380e584b218fd02c8afaf0485d P 2bffd7b80ee6450fa765d1ed59a558c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:59:18.676746 29347 raft_consensus.cc:686] T af75ff380e584b218fd02c8afaf0485d P 2bffd7b80ee6450fa765d1ed59a558c0 [term 1 LEADER]: Becoming Leader. State: Replica: 2bffd7b80ee6450fa765d1ed59a558c0, State: Running, Role: LEADER
I20260929 10:59:18.676807 29347 consensus_queue.cc:237] T af75ff380e584b218fd02c8afaf0485d P 2bffd7b80ee6450fa765d1ed59a558c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } }
I20260929 10:59:18.677508 28833 catalog_manager.cc:5495] T af75ff380e584b218fd02c8afaf0485d P 2bffd7b80ee6450fa765d1ed59a558c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2bffd7b80ee6450fa765d1ed59a558c0 (127.26.120.195). New cstate: current_term: 1 leader_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } health_report { overall_health: HEALTHY } } }
I20260929 10:59:18.727329 29338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:18.725049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45596 (local address 127.26.120.195:44043)
0929 10:59:18.725086 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:18.725088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:18.725097 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:59:18.725165 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:18.725166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:18.725183 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:18.725249 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:59:18.725334 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.725865 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.726643 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.726825 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.726895 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.727038 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:59:18.727159 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:59:18.727216 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:18.727253 (+ 37us) server_negotiation.cc:299] Negotiation successful
0929 10:59:18.727269 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 10:59:18.732417 29339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:18.729916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:53369 (local address 127.26.120.194:41721)
0929 10:59:18.729989 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:18.729990 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:18.730098 (+ 108us) server_negotiation.cc:407] Connection header received
0929 10:59:18.730195 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:18.730196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:18.730207 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:18.730261 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:59:18.730422 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.730894 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.731668 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.731886 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.731955 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.732135 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:59:18.732171 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:59:18.732229 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:18.732308 (+ 79us) server_negotiation.cc:299] Negotiation successful
0929 10:59:18.732334 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 10:59:18.732445 29394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:18.729820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:41721 (local address 127.26.120.195:53369)
0929 10:59:18.730005 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:18.730023 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:18.730124 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:18.730266 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:18.730267 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:18.730268 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 10:59:18.730340 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:18.730341 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.730969 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.730970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:18.731571 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:18.731572 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.731965 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.731966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:18.732029 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.732040 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 10:59:18.732231 (+ 191us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 10:59:18.732234 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:59:18.732302 (+ 68us) client_negotiation.cc:240] Negotiation successful
0929 10:59:18.732345 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":103,"threads_started":1}
I20260929 10:59:18.734793 28991 raft_consensus.cc:1261] T af75ff380e584b218fd02c8afaf0485d P cc42405df36e42629f0e2fd1a75c2751 [term 1 FOLLOWER]: Refusing update from remote peer 2bffd7b80ee6450fa765d1ed59a558c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:59:18.734826 29131 raft_consensus.cc:1261] T af75ff380e584b218fd02c8afaf0485d P 9f3290ac12b64068a3316deb116f06f8 [term 1 FOLLOWER]: Refusing update from remote peer 2bffd7b80ee6450fa765d1ed59a558c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:59:18.735038 29347 consensus_queue.cc:1035] T af75ff380e584b218fd02c8afaf0485d P 2bffd7b80ee6450fa765d1ed59a558c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:59:18.735153 29390 consensus_queue.cc:1035] T af75ff380e584b218fd02c8afaf0485d P 2bffd7b80ee6450fa765d1ed59a558c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:59:18.742329 29338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:18.740091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:40681 (local address 127.26.120.195:44043)
0929 10:59:18.740149 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:18.740151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:18.740158 (+ 7us) server_negotiation.cc:407] Connection header received
0929 10:59:18.740178 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:18.740179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:18.740188 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:18.740242 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:59:18.740385 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.740855 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.741575 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.741856 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.741930 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.742103 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:59:18.742139 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:59:18.742195 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:18.742264 (+ 69us) server_negotiation.cc:299] Negotiation successful
0929 10:59:18.742281 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 10:59:18.742363 29185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:18.739995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:44043 (local address 127.26.120.194:40681)
0929 10:59:18.740033 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:18.740044 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:18.740082 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:18.740265 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:18.740265 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:18.740267 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 10:59:18.740311 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:18.740312 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.740929 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.740930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:18.741470 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:18.741472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:18.741941 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:18.741942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:18.742003 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:18.742009 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 10:59:18.742211 (+ 202us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 10:59:18.742213 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 10:59:18.742260 (+ 47us) client_negotiation.cc:240] Negotiation successful
0929 10:59:18.742302 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
W20260929 10:59:18.742713 29369 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260929 10:59:20.261965 28833 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:59:20.262102 28833 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:59:20.262925 28833 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9ffa6da9b319473ea32390a2c8410b30: Sending DeleteTablet for 3 replicas of tablet af75ff380e584b218fd02c8afaf0485d
I20260929 10:59:20.263190 29110 tablet_service.cc:1514] Processing DeleteTablet for tablet af75ff380e584b218fd02c8afaf0485d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:20 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:46918
I20260929 10:59:20.263239 29252 tablet_service.cc:1514] Processing DeleteTablet for tablet af75ff380e584b218fd02c8afaf0485d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:20 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:45592
I20260929 10:59:20.263365 28971 tablet_service.cc:1514] Processing DeleteTablet for tablet af75ff380e584b218fd02c8afaf0485d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:20 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:60498
I20260929 10:59:20.263541 29421 tablet_replica.cc:330] T af75ff380e584b218fd02c8afaf0485d P cc42405df36e42629f0e2fd1a75c2751: stopping tablet replica
I20260929 10:59:20.263549 29419 tablet_replica.cc:330] T af75ff380e584b218fd02c8afaf0485d P 9f3290ac12b64068a3316deb116f06f8: stopping tablet replica
I20260929 10:59:20.263600 29419 raft_consensus.cc:2229] T af75ff380e584b218fd02c8afaf0485d P 9f3290ac12b64068a3316deb116f06f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:59:20.263646 29419 raft_consensus.cc:2258] T af75ff380e584b218fd02c8afaf0485d P 9f3290ac12b64068a3316deb116f06f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:20.263716 29421 raft_consensus.cc:2229] T af75ff380e584b218fd02c8afaf0485d P cc42405df36e42629f0e2fd1a75c2751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:59:20.263798 29421 raft_consensus.cc:2258] T af75ff380e584b218fd02c8afaf0485d P cc42405df36e42629f0e2fd1a75c2751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:20.263916 29419 ts_tablet_manager.cc:1905] T af75ff380e584b218fd02c8afaf0485d P 9f3290ac12b64068a3316deb116f06f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:20.265407 29420 tablet_replica.cc:330] T af75ff380e584b218fd02c8afaf0485d P 2bffd7b80ee6450fa765d1ed59a558c0: stopping tablet replica
I20260929 10:59:20.265488 29420 raft_consensus.cc:2229] T af75ff380e584b218fd02c8afaf0485d P 2bffd7b80ee6450fa765d1ed59a558c0 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:59:20.265483 29419 ts_tablet_manager.cc:1918] T af75ff380e584b218fd02c8afaf0485d P 9f3290ac12b64068a3316deb116f06f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 10:59:20.265554 29419 log.cc:1196] T af75ff380e584b218fd02c8afaf0485d P 9f3290ac12b64068a3316deb116f06f8: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-1/wal/wals/af75ff380e584b218fd02c8afaf0485d
I20260929 10:59:20.265606 29420 raft_consensus.cc:2258] T af75ff380e584b218fd02c8afaf0485d P 2bffd7b80ee6450fa765d1ed59a558c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:20.265791 29419 ts_tablet_manager.cc:1939] T af75ff380e584b218fd02c8afaf0485d P 9f3290ac12b64068a3316deb116f06f8: Deleting consensus metadata
I20260929 10:59:20.265863 29420 ts_tablet_manager.cc:1905] T af75ff380e584b218fd02c8afaf0485d P 2bffd7b80ee6450fa765d1ed59a558c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:20.266163 29421 ts_tablet_manager.cc:1905] T af75ff380e584b218fd02c8afaf0485d P cc42405df36e42629f0e2fd1a75c2751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:20.267266 29420 ts_tablet_manager.cc:1918] T af75ff380e584b218fd02c8afaf0485d P 2bffd7b80ee6450fa765d1ed59a558c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 10:59:20.267335 29420 log.cc:1196] T af75ff380e584b218fd02c8afaf0485d P 2bffd7b80ee6450fa765d1ed59a558c0: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-2/wal/wals/af75ff380e584b218fd02c8afaf0485d
I20260929 10:59:20.267633 29420 ts_tablet_manager.cc:1939] T af75ff380e584b218fd02c8afaf0485d P 2bffd7b80ee6450fa765d1ed59a558c0: Deleting consensus metadata
I20260929 10:59:20.267665 29421 ts_tablet_manager.cc:1918] T af75ff380e584b218fd02c8afaf0485d P cc42405df36e42629f0e2fd1a75c2751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 10:59:20.267715 29421 log.cc:1196] T af75ff380e584b218fd02c8afaf0485d P cc42405df36e42629f0e2fd1a75c2751: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-0/wal/wals/af75ff380e584b218fd02c8afaf0485d
I20260929 10:59:20.267817 28821 catalog_manager.cc:4841] TS 9f3290ac12b64068a3316deb116f06f8 (127.26.120.194:41721): tablet af75ff380e584b218fd02c8afaf0485d (table test-table [id=b467eaab7c3e48488065bfc3440f9d59]) successfully deleted
I20260929 10:59:20.267964 29421 ts_tablet_manager.cc:1939] T af75ff380e584b218fd02c8afaf0485d P cc42405df36e42629f0e2fd1a75c2751: Deleting consensus metadata
I20260929 10:59:20.267961 28820 catalog_manager.cc:4841] TS 2bffd7b80ee6450fa765d1ed59a558c0 (127.26.120.195:44043): tablet af75ff380e584b218fd02c8afaf0485d (table test-table [id=b467eaab7c3e48488065bfc3440f9d59]) successfully deleted
I20260929 10:59:20.268239 28819 catalog_manager.cc:4841] TS cc42405df36e42629f0e2fd1a75c2751 (127.26.120.193:34187): tablet af75ff380e584b218fd02c8afaf0485d (table test-table [id=b467eaab7c3e48488065bfc3440f9d59]) successfully deleted
Sep 29 10:59:20 dist-test-slave-2lzq krb5kdc[28789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679560, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 10:59:20 dist-test-slave-2lzq krb5kdc[28789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679560, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:20.286886 29429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:20.279884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47794 (local address 127.26.120.254:42121)
0929 10:59:20.280066 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:20.280068 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:20.280076 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:59:20.280099 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:20.280100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:20.280110 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:20.280177 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:20.280702 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:20.281222 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:20.281990 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:20.282250 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:20.282306 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:20.284425 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:20.284427 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:20.284427 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:20.284434 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:20.285726 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:20.286138 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:20.286140 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:20.286140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:20.286204 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:20.286389 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:20.286390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:20.286390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:20.286502 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:20.286585 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:20.286675 (+ 90us) server_negotiation.cc:299] Negotiation successful
0929 10:59:20.286750 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":94,"threads_started":1}
I20260929 10:59:20.288533 28833 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47794:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:59:20.288715 28833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:59:20.290198 29110 tablet_service.cc:1467] Processing CreateTablet for tablet b926d9caf30a42d6978ad614e8d69f20 (DEFAULT_TABLE table=test-table [id=f926f9e7a7c548d9ab14f75d68616272]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:20.290237 29252 tablet_service.cc:1467] Processing CreateTablet for tablet b926d9caf30a42d6978ad614e8d69f20 (DEFAULT_TABLE table=test-table [id=f926f9e7a7c548d9ab14f75d68616272]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:20.290361 29110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b926d9caf30a42d6978ad614e8d69f20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:20.290376 29252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b926d9caf30a42d6978ad614e8d69f20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:20.291376 29430 tablet_bootstrap.cc:492] T b926d9caf30a42d6978ad614e8d69f20 P 2bffd7b80ee6450fa765d1ed59a558c0: Bootstrap starting.
I20260929 10:59:20.291570 28971 tablet_service.cc:1467] Processing CreateTablet for tablet b926d9caf30a42d6978ad614e8d69f20 (DEFAULT_TABLE table=test-table [id=f926f9e7a7c548d9ab14f75d68616272]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:20.291621 29431 tablet_bootstrap.cc:492] T b926d9caf30a42d6978ad614e8d69f20 P 9f3290ac12b64068a3316deb116f06f8: Bootstrap starting.
I20260929 10:59:20.291698 28971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b926d9caf30a42d6978ad614e8d69f20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:20.291841 29430 tablet_bootstrap.cc:654] T b926d9caf30a42d6978ad614e8d69f20 P 2bffd7b80ee6450fa765d1ed59a558c0: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:20.291989 29431 tablet_bootstrap.cc:654] T b926d9caf30a42d6978ad614e8d69f20 P 9f3290ac12b64068a3316deb116f06f8: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:20.292528 29430 tablet_bootstrap.cc:492] T b926d9caf30a42d6978ad614e8d69f20 P 2bffd7b80ee6450fa765d1ed59a558c0: No bootstrap required, opened a new log
I20260929 10:59:20.292570 29430 ts_tablet_manager.cc:1397] T b926d9caf30a42d6978ad614e8d69f20 P 2bffd7b80ee6450fa765d1ed59a558c0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:59:20.292696 29430 raft_consensus.cc:348] T b926d9caf30a42d6978ad614e8d69f20 P 2bffd7b80ee6450fa765d1ed59a558c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } }
I20260929 10:59:20.292766 29430 raft_consensus.cc:374] T b926d9caf30a42d6978ad614e8d69f20 P 2bffd7b80ee6450fa765d1ed59a558c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:20.292780 29430 raft_consensus.cc:729] T b926d9caf30a42d6978ad614e8d69f20 P 2bffd7b80ee6450fa765d1ed59a558c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bffd7b80ee6450fa765d1ed59a558c0, State: Initialized, Role: FOLLOWER
I20260929 10:59:20.292832 29430 consensus_queue.cc:260] T b926d9caf30a42d6978ad614e8d69f20 P 2bffd7b80ee6450fa765d1ed59a558c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } }
I20260929 10:59:20.292927 29430 ts_tablet_manager.cc:1428] T b926d9caf30a42d6978ad614e8d69f20 P 2bffd7b80ee6450fa765d1ed59a558c0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260929 10:59:20.293071 29434 tablet_bootstrap.cc:492] T b926d9caf30a42d6978ad614e8d69f20 P cc42405df36e42629f0e2fd1a75c2751: Bootstrap starting.
I20260929 10:59:20.293427 29434 tablet_bootstrap.cc:654] T b926d9caf30a42d6978ad614e8d69f20 P cc42405df36e42629f0e2fd1a75c2751: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:20.293515 29431 tablet_bootstrap.cc:492] T b926d9caf30a42d6978ad614e8d69f20 P 9f3290ac12b64068a3316deb116f06f8: No bootstrap required, opened a new log
I20260929 10:59:20.293583 29431 ts_tablet_manager.cc:1397] T b926d9caf30a42d6978ad614e8d69f20 P 9f3290ac12b64068a3316deb116f06f8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:59:20.293694 29431 raft_consensus.cc:348] T b926d9caf30a42d6978ad614e8d69f20 P 9f3290ac12b64068a3316deb116f06f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } }
I20260929 10:59:20.293769 29431 raft_consensus.cc:374] T b926d9caf30a42d6978ad614e8d69f20 P 9f3290ac12b64068a3316deb116f06f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:20.293799 29431 raft_consensus.cc:729] T b926d9caf30a42d6978ad614e8d69f20 P 9f3290ac12b64068a3316deb116f06f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f3290ac12b64068a3316deb116f06f8, State: Initialized, Role: FOLLOWER
I20260929 10:59:20.293864 29431 consensus_queue.cc:260] T b926d9caf30a42d6978ad614e8d69f20 P 9f3290ac12b64068a3316deb116f06f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } }
I20260929 10:59:20.294039 29431 ts_tablet_manager.cc:1428] T b926d9caf30a42d6978ad614e8d69f20 P 9f3290ac12b64068a3316deb116f06f8: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260929 10:59:20.294543 29434 tablet_bootstrap.cc:492] T b926d9caf30a42d6978ad614e8d69f20 P cc42405df36e42629f0e2fd1a75c2751: No bootstrap required, opened a new log
I20260929 10:59:20.294612 29434 ts_tablet_manager.cc:1397] T b926d9caf30a42d6978ad614e8d69f20 P cc42405df36e42629f0e2fd1a75c2751: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:59:20.294771 29434 raft_consensus.cc:348] T b926d9caf30a42d6978ad614e8d69f20 P cc42405df36e42629f0e2fd1a75c2751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } }
I20260929 10:59:20.294845 29434 raft_consensus.cc:374] T b926d9caf30a42d6978ad614e8d69f20 P cc42405df36e42629f0e2fd1a75c2751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:20.294878 29434 raft_consensus.cc:729] T b926d9caf30a42d6978ad614e8d69f20 P cc42405df36e42629f0e2fd1a75c2751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc42405df36e42629f0e2fd1a75c2751, State: Initialized, Role: FOLLOWER
I20260929 10:59:20.294948 29434 consensus_queue.cc:260] T b926d9caf30a42d6978ad614e8d69f20 P cc42405df36e42629f0e2fd1a75c2751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } }
I20260929 10:59:20.295125 29434 ts_tablet_manager.cc:1428] T b926d9caf30a42d6978ad614e8d69f20 P cc42405df36e42629f0e2fd1a75c2751: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260929 10:59:20.543841 29397 raft_consensus.cc:482] T b926d9caf30a42d6978ad614e8d69f20 P 2bffd7b80ee6450fa765d1ed59a558c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:59:20.543990 29397 raft_consensus.cc:504] T b926d9caf30a42d6978ad614e8d69f20 P 2bffd7b80ee6450fa765d1ed59a558c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } }
I20260929 10:59:20.544209 29397 leader_election.cc:288] T b926d9caf30a42d6978ad614e8d69f20 P 2bffd7b80ee6450fa765d1ed59a558c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc42405df36e42629f0e2fd1a75c2751 (127.26.120.193:34187), 9f3290ac12b64068a3316deb116f06f8 (127.26.120.194:41721)
I20260929 10:59:20.544407 28991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b926d9caf30a42d6978ad614e8d69f20" candidate_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc42405df36e42629f0e2fd1a75c2751" is_pre_election: true
I20260929 10:59:20.544461 29131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b926d9caf30a42d6978ad614e8d69f20" candidate_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f3290ac12b64068a3316deb116f06f8" is_pre_election: true
I20260929 10:59:20.544525 28991 raft_consensus.cc:2454] T b926d9caf30a42d6978ad614e8d69f20 P cc42405df36e42629f0e2fd1a75c2751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2bffd7b80ee6450fa765d1ed59a558c0 in term 0.
I20260929 10:59:20.544572 29131 raft_consensus.cc:2454] T b926d9caf30a42d6978ad614e8d69f20 P 9f3290ac12b64068a3316deb116f06f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2bffd7b80ee6450fa765d1ed59a558c0 in term 0.
I20260929 10:59:20.544679 29204 leader_election.cc:302] T b926d9caf30a42d6978ad614e8d69f20 P 2bffd7b80ee6450fa765d1ed59a558c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bffd7b80ee6450fa765d1ed59a558c0, cc42405df36e42629f0e2fd1a75c2751; no voters:
I20260929 10:59:20.544795 29397 raft_consensus.cc:2781] T b926d9caf30a42d6978ad614e8d69f20 P 2bffd7b80ee6450fa765d1ed59a558c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:59:20.544869 29397 raft_consensus.cc:482] T b926d9caf30a42d6978ad614e8d69f20 P 2bffd7b80ee6450fa765d1ed59a558c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:59:20.544898 29397 raft_consensus.cc:3037] T b926d9caf30a42d6978ad614e8d69f20 P 2bffd7b80ee6450fa765d1ed59a558c0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:20.545644 29397 raft_consensus.cc:504] T b926d9caf30a42d6978ad614e8d69f20 P 2bffd7b80ee6450fa765d1ed59a558c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } }
I20260929 10:59:20.545795 29397 leader_election.cc:288] T b926d9caf30a42d6978ad614e8d69f20 P 2bffd7b80ee6450fa765d1ed59a558c0 [CANDIDATE]: Term 1 election: Requested vote from peers cc42405df36e42629f0e2fd1a75c2751 (127.26.120.193:34187), 9f3290ac12b64068a3316deb116f06f8 (127.26.120.194:41721)
I20260929 10:59:20.545935 29131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b926d9caf30a42d6978ad614e8d69f20" candidate_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f3290ac12b64068a3316deb116f06f8"
I20260929 10:59:20.546026 29131 raft_consensus.cc:3037] T b926d9caf30a42d6978ad614e8d69f20 P 9f3290ac12b64068a3316deb116f06f8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:20.546083 28991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b926d9caf30a42d6978ad614e8d69f20" candidate_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc42405df36e42629f0e2fd1a75c2751"
I20260929 10:59:20.546185 28991 raft_consensus.cc:3037] T b926d9caf30a42d6978ad614e8d69f20 P cc42405df36e42629f0e2fd1a75c2751 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:20.546706 29131 raft_consensus.cc:2454] T b926d9caf30a42d6978ad614e8d69f20 P 9f3290ac12b64068a3316deb116f06f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2bffd7b80ee6450fa765d1ed59a558c0 in term 1.
I20260929 10:59:20.546876 28991 raft_consensus.cc:2454] T b926d9caf30a42d6978ad614e8d69f20 P cc42405df36e42629f0e2fd1a75c2751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2bffd7b80ee6450fa765d1ed59a558c0 in term 1.
I20260929 10:59:20.546944 29206 leader_election.cc:302] T b926d9caf30a42d6978ad614e8d69f20 P 2bffd7b80ee6450fa765d1ed59a558c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bffd7b80ee6450fa765d1ed59a558c0, 9f3290ac12b64068a3316deb116f06f8; no voters:
I20260929 10:59:20.547029 29397 raft_consensus.cc:2781] T b926d9caf30a42d6978ad614e8d69f20 P 2bffd7b80ee6450fa765d1ed59a558c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:59:20.547070 29397 raft_consensus.cc:686] T b926d9caf30a42d6978ad614e8d69f20 P 2bffd7b80ee6450fa765d1ed59a558c0 [term 1 LEADER]: Becoming Leader. State: Replica: 2bffd7b80ee6450fa765d1ed59a558c0, State: Running, Role: LEADER
I20260929 10:59:20.547116 29397 consensus_queue.cc:237] T b926d9caf30a42d6978ad614e8d69f20 P 2bffd7b80ee6450fa765d1ed59a558c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } }
I20260929 10:59:20.547734 28834 catalog_manager.cc:5495] T b926d9caf30a42d6978ad614e8d69f20 P 2bffd7b80ee6450fa765d1ed59a558c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2bffd7b80ee6450fa765d1ed59a558c0 (127.26.120.195). New cstate: current_term: 1 leader_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } health_report { overall_health: HEALTHY } } }
I20260929 10:59:20.580207 29439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:20.577584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45608 (local address 127.26.120.195:44043)
0929 10:59:20.577746 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:20.577747 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:20.577755 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:59:20.577777 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:20.577778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:20.577810 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:20.577881 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:59:20.578081 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:20.578633 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:20.579482 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:20.579715 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:20.579789 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:20.579934 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:59:20.579969 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:59:20.580033 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:20.580094 (+ 61us) server_negotiation.cc:299] Negotiation successful
0929 10:59:20.580116 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":132,"threads_started":1}
I20260929 10:59:20.582307 29131 raft_consensus.cc:1261] T b926d9caf30a42d6978ad614e8d69f20 P 9f3290ac12b64068a3316deb116f06f8 [term 1 FOLLOWER]: Refusing update from remote peer 2bffd7b80ee6450fa765d1ed59a558c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:59:20.582346 28991 raft_consensus.cc:1261] T b926d9caf30a42d6978ad614e8d69f20 P cc42405df36e42629f0e2fd1a75c2751 [term 1 FOLLOWER]: Refusing update from remote peer 2bffd7b80ee6450fa765d1ed59a558c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:59:20.582567 29411 consensus_queue.cc:1035] T b926d9caf30a42d6978ad614e8d69f20 P 2bffd7b80ee6450fa765d1ed59a558c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:59:20.582710 29397 consensus_queue.cc:1035] T b926d9caf30a42d6978ad614e8d69f20 P 2bffd7b80ee6450fa765d1ed59a558c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:59:20.592199 28834 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:59:20.592311 28834 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:59:20.592947 28834 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9ffa6da9b319473ea32390a2c8410b30: Sending DeleteTablet for 3 replicas of tablet b926d9caf30a42d6978ad614e8d69f20
I20260929 10:59:20.593163 28971 tablet_service.cc:1514] Processing DeleteTablet for tablet b926d9caf30a42d6978ad614e8d69f20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:20 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:60498
I20260929 10:59:20.593187 29252 tablet_service.cc:1514] Processing DeleteTablet for tablet b926d9caf30a42d6978ad614e8d69f20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:20 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:45592
I20260929 10:59:20.593283 29420 tablet_replica.cc:330] T b926d9caf30a42d6978ad614e8d69f20 P 2bffd7b80ee6450fa765d1ed59a558c0: stopping tablet replica
I20260929 10:59:20.593289 29421 tablet_replica.cc:330] T b926d9caf30a42d6978ad614e8d69f20 P cc42405df36e42629f0e2fd1a75c2751: stopping tablet replica
I20260929 10:59:20.593343 29420 raft_consensus.cc:2229] T b926d9caf30a42d6978ad614e8d69f20 P 2bffd7b80ee6450fa765d1ed59a558c0 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:59:20.593343 29421 raft_consensus.cc:2229] T b926d9caf30a42d6978ad614e8d69f20 P cc42405df36e42629f0e2fd1a75c2751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:59:20.593395 29421 raft_consensus.cc:2258] T b926d9caf30a42d6978ad614e8d69f20 P cc42405df36e42629f0e2fd1a75c2751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:20.593407 29420 raft_consensus.cc:2258] T b926d9caf30a42d6978ad614e8d69f20 P 2bffd7b80ee6450fa765d1ed59a558c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:20.593468 28834 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47794:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 10:59:20.593549 29110 tablet_service.cc:1514] Processing DeleteTablet for tablet b926d9caf30a42d6978ad614e8d69f20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:20 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:46918
W20260929 10:59:20.593612 28834 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:59:20.593705 29420 ts_tablet_manager.cc:1905] T b926d9caf30a42d6978ad614e8d69f20 P 2bffd7b80ee6450fa765d1ed59a558c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:20.593707 29419 tablet_replica.cc:330] T b926d9caf30a42d6978ad614e8d69f20 P 9f3290ac12b64068a3316deb116f06f8: stopping tablet replica
I20260929 10:59:20.593765 29419 raft_consensus.cc:2229] T b926d9caf30a42d6978ad614e8d69f20 P 9f3290ac12b64068a3316deb116f06f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:59:20.593803 29419 raft_consensus.cc:2258] T b926d9caf30a42d6978ad614e8d69f20 P 9f3290ac12b64068a3316deb116f06f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:20.593802 29421 ts_tablet_manager.cc:1905] T b926d9caf30a42d6978ad614e8d69f20 P cc42405df36e42629f0e2fd1a75c2751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:20.594319 29419 ts_tablet_manager.cc:1905] T b926d9caf30a42d6978ad614e8d69f20 P 9f3290ac12b64068a3316deb116f06f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:20.594887 28971 tablet_service.cc:1467] Processing CreateTablet for tablet 38625a0758ee4fea9881a01a1d278012 (DEFAULT_TABLE table=test-table [id=fd6f0e31996142c9b861434e9c4e1508]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:20.594982 28971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38625a0758ee4fea9881a01a1d278012. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:20.595333 29110 tablet_service.cc:1467] Processing CreateTablet for tablet 38625a0758ee4fea9881a01a1d278012 (DEFAULT_TABLE table=test-table [id=fd6f0e31996142c9b861434e9c4e1508]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:20.595412 29420 ts_tablet_manager.cc:1918] T b926d9caf30a42d6978ad614e8d69f20 P 2bffd7b80ee6450fa765d1ed59a558c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:59:20.595431 29110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38625a0758ee4fea9881a01a1d278012. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:20.595460 29420 log.cc:1196] T b926d9caf30a42d6978ad614e8d69f20 P 2bffd7b80ee6450fa765d1ed59a558c0: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-2/wal/wals/b926d9caf30a42d6978ad614e8d69f20
I20260929 10:59:20.595709 29420 ts_tablet_manager.cc:1939] T b926d9caf30a42d6978ad614e8d69f20 P 2bffd7b80ee6450fa765d1ed59a558c0: Deleting consensus metadata
I20260929 10:59:20.595850 29419 ts_tablet_manager.cc:1918] T b926d9caf30a42d6978ad614e8d69f20 P 9f3290ac12b64068a3316deb116f06f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:59:20.595906 29419 log.cc:1196] T b926d9caf30a42d6978ad614e8d69f20 P 9f3290ac12b64068a3316deb116f06f8: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-1/wal/wals/b926d9caf30a42d6978ad614e8d69f20
I20260929 10:59:20.596109 29419 ts_tablet_manager.cc:1939] T b926d9caf30a42d6978ad614e8d69f20 P 9f3290ac12b64068a3316deb116f06f8: Deleting consensus metadata
I20260929 10:59:20.596208 28820 catalog_manager.cc:4841] TS 2bffd7b80ee6450fa765d1ed59a558c0 (127.26.120.195:44043): tablet b926d9caf30a42d6978ad614e8d69f20 (table test-table [id=f926f9e7a7c548d9ab14f75d68616272]) successfully deleted
I20260929 10:59:20.596366 28821 catalog_manager.cc:4841] TS 9f3290ac12b64068a3316deb116f06f8 (127.26.120.194:41721): tablet b926d9caf30a42d6978ad614e8d69f20 (table test-table [id=f926f9e7a7c548d9ab14f75d68616272]) successfully deleted
I20260929 10:59:20.596611 29431 tablet_bootstrap.cc:492] T 38625a0758ee4fea9881a01a1d278012 P 9f3290ac12b64068a3316deb116f06f8: Bootstrap starting.
I20260929 10:59:20.596619 29252 tablet_service.cc:1467] Processing CreateTablet for tablet 38625a0758ee4fea9881a01a1d278012 (DEFAULT_TABLE table=test-table [id=fd6f0e31996142c9b861434e9c4e1508]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:20.596731 29252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38625a0758ee4fea9881a01a1d278012. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:20.596882 29431 tablet_bootstrap.cc:654] T 38625a0758ee4fea9881a01a1d278012 P 9f3290ac12b64068a3316deb116f06f8: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:20.597410 29431 tablet_bootstrap.cc:492] T 38625a0758ee4fea9881a01a1d278012 P 9f3290ac12b64068a3316deb116f06f8: No bootstrap required, opened a new log
I20260929 10:59:20.597472 29431 ts_tablet_manager.cc:1397] T 38625a0758ee4fea9881a01a1d278012 P 9f3290ac12b64068a3316deb116f06f8: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 10:59:20.597576 29431 raft_consensus.cc:348] T 38625a0758ee4fea9881a01a1d278012 P 9f3290ac12b64068a3316deb116f06f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } }
I20260929 10:59:20.597635 29430 tablet_bootstrap.cc:492] T 38625a0758ee4fea9881a01a1d278012 P 2bffd7b80ee6450fa765d1ed59a558c0: Bootstrap starting.
I20260929 10:59:20.597645 29431 raft_consensus.cc:374] T 38625a0758ee4fea9881a01a1d278012 P 9f3290ac12b64068a3316deb116f06f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:20.597668 29431 raft_consensus.cc:729] T 38625a0758ee4fea9881a01a1d278012 P 9f3290ac12b64068a3316deb116f06f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f3290ac12b64068a3316deb116f06f8, State: Initialized, Role: FOLLOWER
I20260929 10:59:20.597702 29421 ts_tablet_manager.cc:1918] T b926d9caf30a42d6978ad614e8d69f20 P cc42405df36e42629f0e2fd1a75c2751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:59:20.597751 29421 log.cc:1196] T b926d9caf30a42d6978ad614e8d69f20 P cc42405df36e42629f0e2fd1a75c2751: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-0/wal/wals/b926d9caf30a42d6978ad614e8d69f20
I20260929 10:59:20.597719 29431 consensus_queue.cc:260] T 38625a0758ee4fea9881a01a1d278012 P 9f3290ac12b64068a3316deb116f06f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } }
I20260929 10:59:20.597813 29431 ts_tablet_manager.cc:1428] T 38625a0758ee4fea9881a01a1d278012 P 9f3290ac12b64068a3316deb116f06f8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:20.597927 29421 ts_tablet_manager.cc:1939] T b926d9caf30a42d6978ad614e8d69f20 P cc42405df36e42629f0e2fd1a75c2751: Deleting consensus metadata
I20260929 10:59:20.597941 29430 tablet_bootstrap.cc:654] T 38625a0758ee4fea9881a01a1d278012 P 2bffd7b80ee6450fa765d1ed59a558c0: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:20.598218 28819 catalog_manager.cc:4841] TS cc42405df36e42629f0e2fd1a75c2751 (127.26.120.193:34187): tablet b926d9caf30a42d6978ad614e8d69f20 (table test-table [id=f926f9e7a7c548d9ab14f75d68616272]) successfully deleted
I20260929 10:59:20.598578 29430 tablet_bootstrap.cc:492] T 38625a0758ee4fea9881a01a1d278012 P 2bffd7b80ee6450fa765d1ed59a558c0: No bootstrap required, opened a new log
I20260929 10:59:20.598642 29430 ts_tablet_manager.cc:1397] T 38625a0758ee4fea9881a01a1d278012 P 2bffd7b80ee6450fa765d1ed59a558c0: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 10:59:20.598733 29434 tablet_bootstrap.cc:492] T 38625a0758ee4fea9881a01a1d278012 P cc42405df36e42629f0e2fd1a75c2751: Bootstrap starting.
I20260929 10:59:20.598744 29430 raft_consensus.cc:348] T 38625a0758ee4fea9881a01a1d278012 P 2bffd7b80ee6450fa765d1ed59a558c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } }
I20260929 10:59:20.598810 29430 raft_consensus.cc:374] T 38625a0758ee4fea9881a01a1d278012 P 2bffd7b80ee6450fa765d1ed59a558c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:20.598830 29430 raft_consensus.cc:729] T 38625a0758ee4fea9881a01a1d278012 P 2bffd7b80ee6450fa765d1ed59a558c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bffd7b80ee6450fa765d1ed59a558c0, State: Initialized, Role: FOLLOWER
I20260929 10:59:20.598876 29430 consensus_queue.cc:260] T 38625a0758ee4fea9881a01a1d278012 P 2bffd7b80ee6450fa765d1ed59a558c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } }
I20260929 10:59:20.598961 29430 ts_tablet_manager.cc:1428] T 38625a0758ee4fea9881a01a1d278012 P 2bffd7b80ee6450fa765d1ed59a558c0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:20.599172 29434 tablet_bootstrap.cc:654] T 38625a0758ee4fea9881a01a1d278012 P cc42405df36e42629f0e2fd1a75c2751: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:20.599680 29434 tablet_bootstrap.cc:492] T 38625a0758ee4fea9881a01a1d278012 P cc42405df36e42629f0e2fd1a75c2751: No bootstrap required, opened a new log
I20260929 10:59:20.599717 29434 ts_tablet_manager.cc:1397] T 38625a0758ee4fea9881a01a1d278012 P cc42405df36e42629f0e2fd1a75c2751: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:59:20.599808 29434 raft_consensus.cc:348] T 38625a0758ee4fea9881a01a1d278012 P cc42405df36e42629f0e2fd1a75c2751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } }
I20260929 10:59:20.599853 29434 raft_consensus.cc:374] T 38625a0758ee4fea9881a01a1d278012 P cc42405df36e42629f0e2fd1a75c2751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:20.599885 29434 raft_consensus.cc:729] T 38625a0758ee4fea9881a01a1d278012 P cc42405df36e42629f0e2fd1a75c2751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc42405df36e42629f0e2fd1a75c2751, State: Initialized, Role: FOLLOWER
I20260929 10:59:20.599942 29434 consensus_queue.cc:260] T 38625a0758ee4fea9881a01a1d278012 P cc42405df36e42629f0e2fd1a75c2751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } }
I20260929 10:59:20.600054 29434 ts_tablet_manager.cc:1428] T 38625a0758ee4fea9881a01a1d278012 P cc42405df36e42629f0e2fd1a75c2751: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:20.756276 29436 raft_consensus.cc:482] T 38625a0758ee4fea9881a01a1d278012 P cc42405df36e42629f0e2fd1a75c2751 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:59:20.756418 29436 raft_consensus.cc:504] T 38625a0758ee4fea9881a01a1d278012 P cc42405df36e42629f0e2fd1a75c2751 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } }
I20260929 10:59:20.756629 29436 leader_election.cc:288] T 38625a0758ee4fea9881a01a1d278012 P cc42405df36e42629f0e2fd1a75c2751 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f3290ac12b64068a3316deb116f06f8 (127.26.120.194:41721), 2bffd7b80ee6450fa765d1ed59a558c0 (127.26.120.195:44043)
I20260929 10:59:20.756812 29273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38625a0758ee4fea9881a01a1d278012" candidate_uuid: "cc42405df36e42629f0e2fd1a75c2751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" is_pre_election: true
I20260929 10:59:20.756836 29131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38625a0758ee4fea9881a01a1d278012" candidate_uuid: "cc42405df36e42629f0e2fd1a75c2751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f3290ac12b64068a3316deb116f06f8" is_pre_election: true
I20260929 10:59:20.756924 29131 raft_consensus.cc:2454] T 38625a0758ee4fea9881a01a1d278012 P 9f3290ac12b64068a3316deb116f06f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc42405df36e42629f0e2fd1a75c2751 in term 0.
I20260929 10:59:20.756922 29273 raft_consensus.cc:2454] T 38625a0758ee4fea9881a01a1d278012 P 2bffd7b80ee6450fa765d1ed59a558c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc42405df36e42629f0e2fd1a75c2751 in term 0.
I20260929 10:59:20.757090 28924 leader_election.cc:302] T 38625a0758ee4fea9881a01a1d278012 P cc42405df36e42629f0e2fd1a75c2751 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bffd7b80ee6450fa765d1ed59a558c0, cc42405df36e42629f0e2fd1a75c2751; no voters:
I20260929 10:59:20.757216 29436 raft_consensus.cc:2781] T 38625a0758ee4fea9881a01a1d278012 P cc42405df36e42629f0e2fd1a75c2751 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:59:20.757269 29436 raft_consensus.cc:482] T 38625a0758ee4fea9881a01a1d278012 P cc42405df36e42629f0e2fd1a75c2751 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:59:20.757295 29436 raft_consensus.cc:3037] T 38625a0758ee4fea9881a01a1d278012 P cc42405df36e42629f0e2fd1a75c2751 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:20.758023 29436 raft_consensus.cc:504] T 38625a0758ee4fea9881a01a1d278012 P cc42405df36e42629f0e2fd1a75c2751 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } }
I20260929 10:59:20.758229 29436 leader_election.cc:288] T 38625a0758ee4fea9881a01a1d278012 P cc42405df36e42629f0e2fd1a75c2751 [CANDIDATE]: Term 1 election: Requested vote from peers 9f3290ac12b64068a3316deb116f06f8 (127.26.120.194:41721), 2bffd7b80ee6450fa765d1ed59a558c0 (127.26.120.195:44043)
I20260929 10:59:20.758410 29273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38625a0758ee4fea9881a01a1d278012" candidate_uuid: "cc42405df36e42629f0e2fd1a75c2751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bffd7b80ee6450fa765d1ed59a558c0"
I20260929 10:59:20.758411 29131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38625a0758ee4fea9881a01a1d278012" candidate_uuid: "cc42405df36e42629f0e2fd1a75c2751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f3290ac12b64068a3316deb116f06f8"
I20260929 10:59:20.758486 29131 raft_consensus.cc:3037] T 38625a0758ee4fea9881a01a1d278012 P 9f3290ac12b64068a3316deb116f06f8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:20.758486 29273 raft_consensus.cc:3037] T 38625a0758ee4fea9881a01a1d278012 P 2bffd7b80ee6450fa765d1ed59a558c0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:20.759207 29273 raft_consensus.cc:2454] T 38625a0758ee4fea9881a01a1d278012 P 2bffd7b80ee6450fa765d1ed59a558c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc42405df36e42629f0e2fd1a75c2751 in term 1.
I20260929 10:59:20.759207 29131 raft_consensus.cc:2454] T 38625a0758ee4fea9881a01a1d278012 P 9f3290ac12b64068a3316deb116f06f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc42405df36e42629f0e2fd1a75c2751 in term 1.
I20260929 10:59:20.759383 28925 leader_election.cc:302] T 38625a0758ee4fea9881a01a1d278012 P cc42405df36e42629f0e2fd1a75c2751 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f3290ac12b64068a3316deb116f06f8, cc42405df36e42629f0e2fd1a75c2751; no voters:
I20260929 10:59:20.759487 29436 raft_consensus.cc:2781] T 38625a0758ee4fea9881a01a1d278012 P cc42405df36e42629f0e2fd1a75c2751 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:59:20.759630 29436 raft_consensus.cc:686] T 38625a0758ee4fea9881a01a1d278012 P cc42405df36e42629f0e2fd1a75c2751 [term 1 LEADER]: Becoming Leader. State: Replica: cc42405df36e42629f0e2fd1a75c2751, State: Running, Role: LEADER
I20260929 10:59:20.759711 29436 consensus_queue.cc:237] T 38625a0758ee4fea9881a01a1d278012 P cc42405df36e42629f0e2fd1a75c2751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } }
I20260929 10:59:20.760497 28834 catalog_manager.cc:5495] T 38625a0758ee4fea9881a01a1d278012 P cc42405df36e42629f0e2fd1a75c2751 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc42405df36e42629f0e2fd1a75c2751 (127.26.120.193). New cstate: current_term: 1 leader_uuid: "cc42405df36e42629f0e2fd1a75c2751" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc42405df36e42629f0e2fd1a75c2751" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 34187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 } health_report { overall_health: UNKNOWN } } }
I20260929 10:59:20.783949 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:20.781337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60510 (local address 127.26.120.193:34187)
0929 10:59:20.781511 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:20.781513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:20.781520 (+ 7us) server_negotiation.cc:407] Connection header received
0929 10:59:20.781555 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:20.781556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:20.781589 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:20.781660 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:59:20.781767 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:20.782344 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:20.783268 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:20.783479 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:20.783529 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:20.783659 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:59:20.783763 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:59:20.783818 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:20.783852 (+ 34us) server_negotiation.cc:299] Negotiation successful
0929 10:59:20.783870 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":116,"threads_started":1}
I20260929 10:59:20.788781 29453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:20.786510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:53751 (local address 127.26.120.194:41721)
0929 10:59:20.786639 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:20.786641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:20.786654 (+ 13us) server_negotiation.cc:407] Connection header received
0929 10:59:20.786675 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:20.786676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:20.786686 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:20.786747 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:59:20.786856 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:20.787406 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:20.788183 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:20.788364 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:20.788411 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:20.788585 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:59:20.788611 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:59:20.788648 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:20.788704 (+ 56us) server_negotiation.cc:299] Negotiation successful
0929 10:59:20.788720 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":72,"threads_started":1}
I20260929 10:59:20.788885 29452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:20.786364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:41721 (local address 127.26.120.193:53751)
0929 10:59:20.786560 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:20.786572 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:20.786627 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:20.786765 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:20.786766 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:20.786767 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 10:59:20.786814 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:20.786815 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:20.787471 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:20.787472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:20.788107 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:20.788109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:20.788430 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:20.788430 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:20.788476 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:20.788490 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 10:59:20.788666 (+ 176us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 10:59:20.788667 (+ 1us) client_negotiation.cc:769] Sending connection context
0929 10:59:20.788701 (+ 34us) client_negotiation.cc:240] Negotiation successful
0929 10:59:20.788737 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":83,"threads_started":1}
I20260929 10:59:20.791142 29131 raft_consensus.cc:1261] T 38625a0758ee4fea9881a01a1d278012 P 9f3290ac12b64068a3316deb116f06f8 [term 1 FOLLOWER]: Refusing update from remote peer cc42405df36e42629f0e2fd1a75c2751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:59:20.791126 29273 raft_consensus.cc:1261] T 38625a0758ee4fea9881a01a1d278012 P 2bffd7b80ee6450fa765d1ed59a558c0 [term 1 FOLLOWER]: Refusing update from remote peer cc42405df36e42629f0e2fd1a75c2751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:59:20.791327 29436 consensus_queue.cc:1035] T 38625a0758ee4fea9881a01a1d278012 P cc42405df36e42629f0e2fd1a75c2751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bffd7b80ee6450fa765d1ed59a558c0" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:59:20.791451 29447 consensus_queue.cc:1035] T 38625a0758ee4fea9881a01a1d278012 P cc42405df36e42629f0e2fd1a75c2751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f3290ac12b64068a3316deb116f06f8" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:59:20.798693 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:20.796393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:35519 (local address 127.26.120.193:34187)
0929 10:59:20.796444 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:20.796446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:20.796545 (+ 99us) server_negotiation.cc:407] Connection header received
0929 10:59:20.796608 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:20.796609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:20.796616 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:20.796652 (+ 36us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:59:20.796733 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:20.797179 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:20.798183 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:20.798365 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:20.798419 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:20.798520 (+ 101us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:59:20.798541 (+ 21us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:59:20.798583 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:20.798630 (+ 47us) server_negotiation.cc:299] Negotiation successful
0929 10:59:20.798645 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260929 10:59:20.798713 29457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:20.796341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:34187 (local address 127.26.120.194:35519)
0929 10:59:20.796488 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:20.796499 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:20.796558 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:20.796661 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:20.796662 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:20.796663 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 10:59:20.796699 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:20.796700 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:20.797238 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:20.797238 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:20.798114 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:20.798117 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:20.798427 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:20.798427 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:20.798466 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:20.798470 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 10:59:20.798584 (+ 114us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 10:59:20.798585 (+ 1us) client_negotiation.cc:769] Sending connection context
0929 10:59:20.798626 (+ 41us) client_negotiation.cc:240] Negotiation successful
0929 10:59:20.798658 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":109,"threads_started":1}
W20260929 10:59:20.799291 29403 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260929 10:59:22.321446 28833 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:59:22.321560 28833 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:59:22.322357 28833 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9ffa6da9b319473ea32390a2c8410b30: Sending DeleteTablet for 3 replicas of tablet 38625a0758ee4fea9881a01a1d278012
I20260929 10:59:22.322602 28971 tablet_service.cc:1514] Processing DeleteTablet for tablet 38625a0758ee4fea9881a01a1d278012 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:22 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:60498
I20260929 10:59:22.322623 29110 tablet_service.cc:1514] Processing DeleteTablet for tablet 38625a0758ee4fea9881a01a1d278012 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:22 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:46918
I20260929 10:59:22.322649 29252 tablet_service.cc:1514] Processing DeleteTablet for tablet 38625a0758ee4fea9881a01a1d278012 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:22 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:45592
I20260929 10:59:22.322834 29485 tablet_replica.cc:330] T 38625a0758ee4fea9881a01a1d278012 P cc42405df36e42629f0e2fd1a75c2751: stopping tablet replica
I20260929 10:59:22.322911 29485 raft_consensus.cc:2229] T 38625a0758ee4fea9881a01a1d278012 P cc42405df36e42629f0e2fd1a75c2751 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:59:22.322918 29486 tablet_replica.cc:330] T 38625a0758ee4fea9881a01a1d278012 P 2bffd7b80ee6450fa765d1ed59a558c0: stopping tablet replica
I20260929 10:59:22.322939 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 28906
I20260929 10:59:22.322995 29485 raft_consensus.cc:2258] T 38625a0758ee4fea9881a01a1d278012 P cc42405df36e42629f0e2fd1a75c2751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:22.323630 29486 raft_consensus.cc:2229] T 38625a0758ee4fea9881a01a1d278012 P 2bffd7b80ee6450fa765d1ed59a558c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:59:22.323693 29486 raft_consensus.cc:2258] T 38625a0758ee4fea9881a01a1d278012 P 2bffd7b80ee6450fa765d1ed59a558c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:22.323879 29484 tablet_replica.cc:330] T 38625a0758ee4fea9881a01a1d278012 P 9f3290ac12b64068a3316deb116f06f8: stopping tablet replica
I20260929 10:59:22.323951 29484 raft_consensus.cc:2229] T 38625a0758ee4fea9881a01a1d278012 P 9f3290ac12b64068a3316deb116f06f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:59:22.323961 29486 ts_tablet_manager.cc:1905] T 38625a0758ee4fea9881a01a1d278012 P 2bffd7b80ee6450fa765d1ed59a558c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:22.323993 29484 raft_consensus.cc:2258] T 38625a0758ee4fea9881a01a1d278012 P 9f3290ac12b64068a3316deb116f06f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:22.324234 29484 ts_tablet_manager.cc:1905] T 38625a0758ee4fea9881a01a1d278012 P 9f3290ac12b64068a3316deb116f06f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:22.325814 29484 ts_tablet_manager.cc:1918] T 38625a0758ee4fea9881a01a1d278012 P 9f3290ac12b64068a3316deb116f06f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 10:59:22.325893 29484 log.cc:1196] T 38625a0758ee4fea9881a01a1d278012 P 9f3290ac12b64068a3316deb116f06f8: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-1/wal/wals/38625a0758ee4fea9881a01a1d278012
I20260929 10:59:22.325901 29486 ts_tablet_manager.cc:1918] T 38625a0758ee4fea9881a01a1d278012 P 2bffd7b80ee6450fa765d1ed59a558c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 10:59:22.325953 29486 log.cc:1196] T 38625a0758ee4fea9881a01a1d278012 P 2bffd7b80ee6450fa765d1ed59a558c0: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790679549986039-27107-0/minicluster-data/ts-2/wal/wals/38625a0758ee4fea9881a01a1d278012
I20260929 10:59:22.326218 29484 ts_tablet_manager.cc:1939] T 38625a0758ee4fea9881a01a1d278012 P 9f3290ac12b64068a3316deb116f06f8: Deleting consensus metadata
I20260929 10:59:22.326256 29486 ts_tablet_manager.cc:1939] T 38625a0758ee4fea9881a01a1d278012 P 2bffd7b80ee6450fa765d1ed59a558c0: Deleting consensus metadata
I20260929 10:59:22.326572 28821 catalog_manager.cc:4841] TS 9f3290ac12b64068a3316deb116f06f8 (127.26.120.194:41721): tablet 38625a0758ee4fea9881a01a1d278012 (table test-table [id=fd6f0e31996142c9b861434e9c4e1508]) successfully deleted
I20260929 10:59:22.326664 28820 catalog_manager.cc:4841] TS 2bffd7b80ee6450fa765d1ed59a558c0 (127.26.120.195:44043): tablet 38625a0758ee4fea9881a01a1d278012 (table test-table [id=fd6f0e31996142c9b861434e9c4e1508]) successfully deleted
W20260929 10:59:22.330466 28819 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.120.193:34187 (error 108)
I20260929 10:59:22.330701 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 29046
I20260929 10:59:22.331487 29488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:22.330851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:34187 (local address 127.0.0.1:60514)
0929 10:59:22.331401 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:22.331452 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.193:34187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":55,"threads_started":1}
W20260929 10:59:22.331616 28819 catalog_manager.cc:4584] TS cc42405df36e42629f0e2fd1a75c2751 (127.26.120.193:34187): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 38625a0758ee4fea9881a01a1d278012: Network error: Client connection negotiation failed: client connection to 127.26.120.193:34187: connect: Connection refused (error 111)
I20260929 10:59:22.337647 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 29187
I20260929 10:59:22.342747 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 28802
2026-09-29T10:59:22Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5995 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:59:22 dist-test-slave-2lzq krb5kdc[29491](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:59:22 dist-test-slave-2lzq krb5kdc[29491](info): set up 2 sockets
Sep 29 10:59:22 dist-test-slave-2lzq krb5kdc[29491](info): commencing operation
krb5kdc: starting...
W20260929 10:59:23.386014 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 10:59:23 dist-test-slave-2lzq krb5kdc[29491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:59:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:59:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.120.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
I20260929 10:59:23.502818 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:44701
--webserver_interface=127.26.120.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:38815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:44701
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:23.581624 29504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:23.581875 29504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:23.581912 29504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:23.583101 29504 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:59:23.583173 29504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:23.583196 29504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:23.583217 29504 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:59:23.583247 29504 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:59:23.584744 29504 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:38815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:44701
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:44701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:23.585026 29504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:23.585258 29504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:23.590343 29509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:23.590411 29504 server_base.cc:1034] running on GCE node
W20260929 10:59:23.590445 29510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:23.590335 29512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:23.590762 29504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:23.591045 29504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:23.592214 29504 hybrid_clock.cc:648] HybridClock initialized: now 1790679563592202 us; error 30 us; skew 500 ppm
Sep 29 10:59:23 dist-test-slave-2lzq krb5kdc[29491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679563, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:23.594702 29504 init.cc:377] Logged in from keytab as kudu/127.26.120.254@KRBTEST.COM (short username kudu)
I20260929 10:59:23.595146 29504 webserver.cc:460] Webserver started at http://127.26.120.254:33881/ using document root <none> and password file <none>
I20260929 10:59:23.595381 29504 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:23.595448 29504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:23.595599 29504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:23.596619 29504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "54cbc646e3064464bdd9b37bebb206ce"
format_stamp: "Formatted at 2026-09-29 10:59:23 on dist-test-slave-2lzq"
server_key: "c680f76cb69b01e76010d02a8e8887bd"
server_key_iv: "96d5cf62ddfff1d830e21daba2049b4d"
server_key_version: "encryptionkey@0"
I20260929 10:59:23.596982 29504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "54cbc646e3064464bdd9b37bebb206ce"
format_stamp: "Formatted at 2026-09-29 10:59:23 on dist-test-slave-2lzq"
server_key: "c680f76cb69b01e76010d02a8e8887bd"
server_key_iv: "96d5cf62ddfff1d830e21daba2049b4d"
server_key_version: "encryptionkey@0"
I20260929 10:59:23.598343 29504 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 10:59:23.599035 29519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:23.599210 29504 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:59:23.599304 29504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "54cbc646e3064464bdd9b37bebb206ce"
format_stamp: "Formatted at 2026-09-29 10:59:23 on dist-test-slave-2lzq"
server_key: "c680f76cb69b01e76010d02a8e8887bd"
server_key_iv: "96d5cf62ddfff1d830e21daba2049b4d"
server_key_version: "encryptionkey@0"
I20260929 10:59:23.599436 29504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:23.616822 29504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:23.619154 29504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:23.619318 29504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:23.623299 29504 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:44701
I20260929 10:59:23.623337 29571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:44701 every 8 connection(s)
I20260929 10:59:23.623700 29504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 10:59:23.624518 29572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:23.626744 29572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54cbc646e3064464bdd9b37bebb206ce: Bootstrap starting.
I20260929 10:59:23.627265 29572 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54cbc646e3064464bdd9b37bebb206ce: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:23.627440 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 29504
I20260929 10:59:23.627535 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 10:59:23.627624 29572 log.cc:824] T 00000000000000000000000000000000 P 54cbc646e3064464bdd9b37bebb206ce: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:23.627703 27107 external_mini_cluster.cc:1428] Setting key ecaadd469cb12bcd4a3afa00a4a2ad97
I20260929 10:59:23.628417 29572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54cbc646e3064464bdd9b37bebb206ce: No bootstrap required, opened a new log
I20260929 10:59:23.630976 29572 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54cbc646e3064464bdd9b37bebb206ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54cbc646e3064464bdd9b37bebb206ce" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44701 } }
I20260929 10:59:23.631145 29572 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54cbc646e3064464bdd9b37bebb206ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:23.631179 29572 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54cbc646e3064464bdd9b37bebb206ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54cbc646e3064464bdd9b37bebb206ce, State: Initialized, Role: FOLLOWER
I20260929 10:59:23.631306 29572 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54cbc646e3064464bdd9b37bebb206ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54cbc646e3064464bdd9b37bebb206ce" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44701 } }
I20260929 10:59:23.631373 29572 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54cbc646e3064464bdd9b37bebb206ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:59:23.631400 29572 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54cbc646e3064464bdd9b37bebb206ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:59:23.631443 29572 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54cbc646e3064464bdd9b37bebb206ce [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:23.632071 29572 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54cbc646e3064464bdd9b37bebb206ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54cbc646e3064464bdd9b37bebb206ce" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44701 } }
I20260929 10:59:23.632158 29572 leader_election.cc:302] T 00000000000000000000000000000000 P 54cbc646e3064464bdd9b37bebb206ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54cbc646e3064464bdd9b37bebb206ce; no voters:
I20260929 10:59:23.632408 29572 leader_election.cc:288] T 00000000000000000000000000000000 P 54cbc646e3064464bdd9b37bebb206ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:59:23.632434 29577 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54cbc646e3064464bdd9b37bebb206ce [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:59:23.632639 29577 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54cbc646e3064464bdd9b37bebb206ce [term 1 LEADER]: Becoming Leader. State: Replica: 54cbc646e3064464bdd9b37bebb206ce, State: Running, Role: LEADER
I20260929 10:59:23.632740 29577 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54cbc646e3064464bdd9b37bebb206ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54cbc646e3064464bdd9b37bebb206ce" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44701 } }
I20260929 10:59:23.632767 29572 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54cbc646e3064464bdd9b37bebb206ce [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:59:23.633529 29578 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54cbc646e3064464bdd9b37bebb206ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54cbc646e3064464bdd9b37bebb206ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54cbc646e3064464bdd9b37bebb206ce" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44701 } } }
I20260929 10:59:23.633639 29578 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54cbc646e3064464bdd9b37bebb206ce [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:23.633805 29579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54cbc646e3064464bdd9b37bebb206ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54cbc646e3064464bdd9b37bebb206ce. Latest consensus state: current_term: 1 leader_uuid: "54cbc646e3064464bdd9b37bebb206ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54cbc646e3064464bdd9b37bebb206ce" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44701 } } }
I20260929 10:59:23.633849 29587 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:59:23.634421 29587 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 29 10:59:23 dist-test-slave-2lzq krb5kdc[29491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:23.633908 29579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54cbc646e3064464bdd9b37bebb206ce [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:23.636219 29587 catalog_manager.cc:1347] Generated new cluster ID: 46dc7d5796e34265a28b90c9a15c05e5
I20260929 10:59:23.636273 29587 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:59:23.638406 29576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:23.628192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37270 (local address 127.26.120.254:44701)
0929 10:59:23.628356 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:23.628365 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:23.628400 (+ 35us) server_negotiation.cc:407] Connection header received
0929 10:59:23.628854 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:23.628861 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:23.628905 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:23.629027 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:23.629756 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:23.630689 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:23.631748 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:23.632169 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:23.632278 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:23.635020 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:23.635024 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:23.635024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:23.635036 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:23.636876 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:23.637243 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:23.637245 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:23.637246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:23.637343 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:23.637556 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:23.637557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:23.637557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:23.637783 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:23.637947 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:23.638027 (+ 80us) server_negotiation.cc:299] Negotiation successful
0929 10:59:23.638227 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":94,"threads_started":1}
I20260929 10:59:23.645440 29587 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:59:23.646160 29587 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:59:23.650274 29587 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54cbc646e3064464bdd9b37bebb206ce: Generated new TSK 0
I20260929 10:59:23.650431 29587 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.120.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab.
Entry for principal kudu/127.26.120.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab.
I20260929 10:59:23.677337 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.193:0
--local_ip_for_outbound_sockets=127.26.120.193
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:44701
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:38815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:23.758127 29598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:23.758314 29598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:23.758337 29598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:23.759476 29598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:23.759526 29598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:23.759552 29598 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.193
I20260929 10:59:23.760916 29598 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:38815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:44701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29598
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.193
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:23.761142 29598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:23.761368 29598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:23.766314 29604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:23.766371 29606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:23.766371 29603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:23.766588 29598 server_base.cc:1034] running on GCE node
I20260929 10:59:23.766798 29598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:23.767046 29598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:23.768199 29598 hybrid_clock.cc:648] HybridClock initialized: now 1790679563768185 us; error 30 us; skew 500 ppm
Sep 29 10:59:23 dist-test-slave-2lzq krb5kdc[29491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679563, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:23.770692 29598 init.cc:377] Logged in from keytab as kudu/127.26.120.193@KRBTEST.COM (short username kudu)
I20260929 10:59:23.771134 29598 webserver.cc:460] Webserver started at http://127.26.120.193:46559/ using document root <none> and password file <none>
I20260929 10:59:23.771394 29598 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:23.771461 29598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:23.771623 29598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:23.772713 29598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-0/data/instance:
uuid: "5d749f0551884673a55acceb3cf3a7cc"
format_stamp: "Formatted at 2026-09-29 10:59:23 on dist-test-slave-2lzq"
server_key: "34f7d89d9d36c165f3a367438b29e722"
server_key_iv: "e53b2768b0e56b4da13a4817b5ee4513"
server_key_version: "encryptionkey@0"
I20260929 10:59:23.773070 29598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance:
uuid: "5d749f0551884673a55acceb3cf3a7cc"
format_stamp: "Formatted at 2026-09-29 10:59:23 on dist-test-slave-2lzq"
server_key: "34f7d89d9d36c165f3a367438b29e722"
server_key_iv: "e53b2768b0e56b4da13a4817b5ee4513"
server_key_version: "encryptionkey@0"
I20260929 10:59:23.774382 29598 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 10:59:23.775075 29614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:23.775250 29598 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:59:23.775341 29598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-0/wal
uuid: "5d749f0551884673a55acceb3cf3a7cc"
format_stamp: "Formatted at 2026-09-29 10:59:23 on dist-test-slave-2lzq"
server_key: "34f7d89d9d36c165f3a367438b29e722"
server_key_iv: "e53b2768b0e56b4da13a4817b5ee4513"
server_key_version: "encryptionkey@0"
I20260929 10:59:23.775426 29598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:23.784802 29598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:23.787079 29598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:23.787235 29598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:23.787456 29598 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:59:23.787752 29598 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:23.787806 29598 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:23.787843 29598 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:23.787871 29598 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:23.793702 29598 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.193:35325
I20260929 10:59:23.793743 29727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.193:35325 every 8 connection(s)
I20260929 10:59:23.794140 29598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
Sep 29 10:59:23 dist-test-slave-2lzq krb5kdc[29491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679563, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:23.802399 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 29598
I20260929 10:59:23.802505 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance
I20260929 10:59:23.802682 27107 external_mini_cluster.cc:1428] Setting key 1eddf2b7b71ceb4fd9894d69a103cd08
I20260929 10:59:23.803447 29576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:23.794846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:54695 (local address 127.26.120.254:44701)
0929 10:59:23.794892 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:23.794895 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:23.795074 (+ 179us) server_negotiation.cc:407] Connection header received
0929 10:59:23.795612 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:23.795613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:23.795626 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:23.795686 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:23.796585 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:23.797112 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:23.798210 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:23.798433 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:23.798508 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:23.800606 (+ 2098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:23.800608 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:23.800608 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:23.800616 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:23.801793 (+ 1177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:23.802322 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:23.802323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:23.802324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:23.802385 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:23.802679 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:23.802679 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:23.802680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:23.802829 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:23.802908 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:23.803251 (+ 343us) server_negotiation.cc:299] Negotiation successful
0929 10:59:23.803358 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 10:59:23.803632 29730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:23.794804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44701 (local address 127.26.120.193:54695)
0929 10:59:23.794941 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:23.794989 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:23.795484 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:23.795754 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:23.795758 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:23.796295 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:23.796477 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:23.796481 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:23.797208 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:23.797209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:23.798020 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:23.798022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:23.798544 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:23.798545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:23.798666 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:23.799164 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:23.799172 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:23.800509 (+ 1337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:23.801888 (+ 1379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:23.801895 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:23.801897 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:23.802248 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:23.802459 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:23.802460 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:23.802461 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:23.802608 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:23.802900 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:23.802908 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:23.803095 (+ 187us) client_negotiation.cc:769] Sending connection context
0929 10:59:23.803227 (+ 132us) client_negotiation.cc:240] Negotiation successful
0929 10:59:23.803381 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":68,"threads_started":1}
I20260929 10:59:23.804122 29728 heartbeater.cc:347] Connected to a master server at 127.26.120.254:44701
I20260929 10:59:23.804219 29728 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:23.804399 29728 heartbeater.cc:511] Master 127.26.120.254:44701 requested a full tablet report, sending...
I20260929 10:59:23.804905 29536 ts_manager.cc:194] Registered new tserver with Master: 5d749f0551884673a55acceb3cf3a7cc (127.26.120.193:35325)
I20260929 10:59:23.805505 29536 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.193@KRBTEST.COM'} at 127.26.120.193:54695
WARNING: no policy specified for kudu/127.26.120.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab.
Entry for principal kudu/127.26.120.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab.
I20260929 10:59:23.827163 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.194:0
--local_ip_for_outbound_sockets=127.26.120.194
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:44701
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:38815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:23.908069 29733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:23.908269 29733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:23.908293 29733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:23.909394 29733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:23.909425 29733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:23.909451 29733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.194
I20260929 10:59:23.910851 29733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:38815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:44701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.194
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:23.911026 29733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:23.911201 29733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:23.915941 29738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:23.915958 29741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:23.915946 29739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:23.916112 29733 server_base.cc:1034] running on GCE node
I20260929 10:59:23.916373 29733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:23.916625 29733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:23.917804 29733 hybrid_clock.cc:648] HybridClock initialized: now 1790679563917786 us; error 34 us; skew 500 ppm
Sep 29 10:59:23 dist-test-slave-2lzq krb5kdc[29491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679563, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:23.920213 29733 init.cc:377] Logged in from keytab as kudu/127.26.120.194@KRBTEST.COM (short username kudu)
I20260929 10:59:23.920634 29733 webserver.cc:460] Webserver started at http://127.26.120.194:43969/ using document root <none> and password file <none>
I20260929 10:59:23.920871 29733 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:23.920939 29733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:23.921092 29733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:23.922184 29733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-1/data/instance:
uuid: "6fe481974ff0421faa1c0866c8a152b6"
format_stamp: "Formatted at 2026-09-29 10:59:23 on dist-test-slave-2lzq"
server_key: "c6e09677774d53512daf139617b11fcb"
server_key_iv: "cc699c2f976a902000cd2bd1e9013412"
server_key_version: "encryptionkey@0"
I20260929 10:59:23.922540 29733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance:
uuid: "6fe481974ff0421faa1c0866c8a152b6"
format_stamp: "Formatted at 2026-09-29 10:59:23 on dist-test-slave-2lzq"
server_key: "c6e09677774d53512daf139617b11fcb"
server_key_iv: "cc699c2f976a902000cd2bd1e9013412"
server_key_version: "encryptionkey@0"
I20260929 10:59:23.923805 29733 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:59:23.924526 29748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:23.924691 29733 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:59:23.924780 29733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-1/wal
uuid: "6fe481974ff0421faa1c0866c8a152b6"
format_stamp: "Formatted at 2026-09-29 10:59:23 on dist-test-slave-2lzq"
server_key: "c6e09677774d53512daf139617b11fcb"
server_key_iv: "cc699c2f976a902000cd2bd1e9013412"
server_key_version: "encryptionkey@0"
I20260929 10:59:23.924860 29733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:24.020253 29733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:24.022228 29733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:24.022400 29733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:24.022665 29733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:59:24.022989 29733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:24.023049 29733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:24.023087 29733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:24.023136 29733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:24.028972 29733 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.194:37733
I20260929 10:59:24.029016 29862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.194:37733 every 8 connection(s)
I20260929 10:59:24.029350 29733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
I20260929 10:59:24.033058 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 29733
I20260929 10:59:24.033159 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance
I20260929 10:59:24.033360 27107 external_mini_cluster.cc:1428] Setting key eccabc5d5d67797b078539bc3d9b35e1
Sep 29 10:59:24 dist-test-slave-2lzq krb5kdc[29491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679563, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:24.039695 29865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:24.030087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44701 (local address 127.26.120.194:33273)
0929 10:59:24.030251 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:24.030286 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:24.030783 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:24.031044 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:24.031047 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:24.031620 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:24.031808 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:24.031812 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:24.032594 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:24.032595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:24.033818 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:24.033821 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:24.034286 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:24.034288 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:24.034451 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:24.034918 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:24.034928 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:24.036453 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:24.038366 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:24.038371 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:24.038373 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:24.038591 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:24.038814 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:24.038814 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:24.038814 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:24.038908 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:24.039202 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:24.039206 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:24.039334 (+ 128us) client_negotiation.cc:769] Sending connection context
0929 10:59:24.039417 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 10:59:24.039536 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":76,"threads_started":1}
I20260929 10:59:24.039705 29576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:24.030107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:33273 (local address 127.26.120.254:44701)
0929 10:59:24.030175 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:24.030191 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:24.030364 (+ 173us) server_negotiation.cc:407] Connection header received
0929 10:59:24.030897 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:24.030899 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:24.030911 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:24.030958 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:24.031906 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:24.032502 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:24.033912 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:24.034204 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:24.034259 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:24.036554 (+ 2295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:24.036556 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:24.036556 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:24.036564 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:24.038287 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:24.038668 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:24.038670 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:24.038670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:24.038739 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:24.038971 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:24.038972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:24.038972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:24.039132 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:24.039212 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:24.039467 (+ 255us) server_negotiation.cc:299] Negotiation successful
0929 10:59:24.039566 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 10:59:24.040163 29863 heartbeater.cc:347] Connected to a master server at 127.26.120.254:44701
I20260929 10:59:24.040271 29863 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:24.040453 29863 heartbeater.cc:511] Master 127.26.120.254:44701 requested a full tablet report, sending...
I20260929 10:59:24.040863 29536 ts_manager.cc:194] Registered new tserver with Master: 6fe481974ff0421faa1c0866c8a152b6 (127.26.120.194:37733)
I20260929 10:59:24.041379 29536 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.194@KRBTEST.COM'} at 127.26.120.194:33273
WARNING: no policy specified for kudu/127.26.120.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab.
Entry for principal kudu/127.26.120.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab.
I20260929 10:59:24.057579 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.195:0
--local_ip_for_outbound_sockets=127.26.120.195
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:44701
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:38815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:24.138700 29868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:24.138916 29868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:24.138940 29868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:24.140738 29868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:24.140815 29868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:24.140856 29868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.195
I20260929 10:59:24.142982 29868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:38815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:44701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.195
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:24.143255 29868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:24.143499 29868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 10:59:24.148437 29868 server_base.cc:1034] running on GCE node
W20260929 10:59:24.148448 29876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:24.148415 29873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:24.148453 29874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:24.148923 29868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:24.149165 29868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:24.150364 29868 hybrid_clock.cc:648] HybridClock initialized: now 1790679564150304 us; error 27 us; skew 500 ppm
Sep 29 10:59:24 dist-test-slave-2lzq krb5kdc[29491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679564, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:24.152778 29868 init.cc:377] Logged in from keytab as kudu/127.26.120.195@KRBTEST.COM (short username kudu)
I20260929 10:59:24.153223 29868 webserver.cc:460] Webserver started at http://127.26.120.195:44231/ using document root <none> and password file <none>
I20260929 10:59:24.153499 29868 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:24.153544 29868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:24.153685 29868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:24.154738 29868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-2/data/instance:
uuid: "bffe49a79bfe46c49810d13675d4c021"
format_stamp: "Formatted at 2026-09-29 10:59:24 on dist-test-slave-2lzq"
server_key: "292295aba5a55ec8231aaf861d2e057d"
server_key_iv: "e46f41a828e2e0cbcf3325d8a3b2bd9e"
server_key_version: "encryptionkey@0"
I20260929 10:59:24.155072 29868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance:
uuid: "bffe49a79bfe46c49810d13675d4c021"
format_stamp: "Formatted at 2026-09-29 10:59:24 on dist-test-slave-2lzq"
server_key: "292295aba5a55ec8231aaf861d2e057d"
server_key_iv: "e46f41a828e2e0cbcf3325d8a3b2bd9e"
server_key_version: "encryptionkey@0"
I20260929 10:59:24.156342 29868 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 10:59:24.156983 29883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:24.157155 29868 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:59:24.157248 29868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-2/wal
uuid: "bffe49a79bfe46c49810d13675d4c021"
format_stamp: "Formatted at 2026-09-29 10:59:24 on dist-test-slave-2lzq"
server_key: "292295aba5a55ec8231aaf861d2e057d"
server_key_iv: "e46f41a828e2e0cbcf3325d8a3b2bd9e"
server_key_version: "encryptionkey@0"
I20260929 10:59:24.157356 29868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:24.181666 29868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:24.184231 29868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:24.184406 29868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:24.184659 29868 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:59:24.184988 29868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:24.185045 29868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:24.185086 29868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:24.185127 29868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:24.191051 29868 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.195:41125
I20260929 10:59:24.191103 29996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.195:41125 every 8 connection(s)
I20260929 10:59:24.191421 29868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
I20260929 10:59:24.192261 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 29868
I20260929 10:59:24.192348 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance
I20260929 10:59:24.192533 27107 external_mini_cluster.cc:1428] Setting key 0308bf818f8f74e2093085ac37042f57
Sep 29 10:59:24 dist-test-slave-2lzq krb5kdc[29491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679564, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:24.200263 29576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:24.192133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:38607 (local address 127.26.120.254:44701)
0929 10:59:24.192186 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:24.192189 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:24.192436 (+ 247us) server_negotiation.cc:407] Connection header received
0929 10:59:24.193102 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:24.193103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:24.193116 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:24.193173 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:24.194261 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:24.194756 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:24.195691 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:24.195873 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:24.195925 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:24.197933 (+ 2008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:24.197934 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:24.197935 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:24.197941 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:24.199170 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:24.199513 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:24.199514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:24.199515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:24.199562 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:24.199760 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:24.199760 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:24.199760 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:24.199860 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:24.199909 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:24.200138 (+ 229us) server_negotiation.cc:299] Negotiation successful
0929 10:59:24.200197 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 10:59:24.200516 29999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:24.192110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44701 (local address 127.26.120.195:38607)
0929 10:59:24.192278 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:24.192329 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:24.193003 (+ 674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:24.193245 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:24.193249 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:24.193890 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:24.194179 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:24.194185 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:24.194830 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:24.194830 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:24.195619 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:24.195621 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:24.195949 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:24.195950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:24.196078 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:24.196526 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:24.196534 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:24.197852 (+ 1318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:24.199237 (+ 1385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:24.199243 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:24.199245 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:24.199453 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:24.199617 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:24.199618 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:24.199618 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:24.199710 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:24.199910 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:24.199914 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:24.200035 (+ 121us) client_negotiation.cc:769] Sending connection context
0929 10:59:24.200116 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 10:59:24.200230 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
I20260929 10:59:24.201020 29997 heartbeater.cc:347] Connected to a master server at 127.26.120.254:44701
I20260929 10:59:24.201098 29997 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:24.201232 29997 heartbeater.cc:511] Master 127.26.120.254:44701 requested a full tablet report, sending...
I20260929 10:59:24.201593 29536 ts_manager.cc:194] Registered new tserver with Master: bffe49a79bfe46c49810d13675d4c021 (127.26.120.195:41125)
I20260929 10:59:24.202044 29536 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.195@KRBTEST.COM'} at 127.26.120.195:38607
I20260929 10:59:24.204046 27107 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 10:59:24.215292 29576 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:24.214522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37286 (local address 127.26.120.254:44701)
0929 10:59:24.214575 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:24.214577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:24.214676 (+ 99us) server_negotiation.cc:407] Connection header received
0929 10:59:24.214716 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:24.214717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:24.214726 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:24.214782 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:24.215242 (+ 460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37286: BlockingRecv error: recv got EOF from 127.0.0.1:37286 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
I20260929 10:59:24.216248 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 29598
I20260929 10:59:24.220777 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 29733
I20260929 10:59:24.225189 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 29868
I20260929 10:59:24.229300 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 29504
2026-09-29T10:59:24Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1885 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:59:24 dist-test-slave-2lzq krb5kdc[30009](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:59:24 dist-test-slave-2lzq krb5kdc[30009](info): set up 2 sockets
Sep 29 10:59:24 dist-test-slave-2lzq krb5kdc[30009](info): commencing operation
krb5kdc: starting...
W20260929 10:59:25.249908 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.001s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 10:59:25 dist-test-slave-2lzq krb5kdc[30009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:59:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:59:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.120.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
I20260929 10:59:25.363070 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:34291
--webserver_interface=127.26.120.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:42659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:34291
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:25.442437 30022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:25.442649 30022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:25.442672 30022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:25.443845 30022 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:59:25.443888 30022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:25.443902 30022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:25.443915 30022 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:59:25.443928 30022 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:59:25.445428 30022 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:42659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:34291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:34291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:25.445696 30022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:25.445928 30022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:25.451313 30028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:25.451328 30030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:25.451327 30027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:25.451617 30022 server_base.cc:1034] running on GCE node
I20260929 10:59:25.451848 30022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:25.452143 30022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:25.453310 30022 hybrid_clock.cc:648] HybridClock initialized: now 1790679565453304 us; error 40 us; skew 500 ppm
Sep 29 10:59:25 dist-test-slave-2lzq krb5kdc[30009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679565, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:25.455726 30022 init.cc:377] Logged in from keytab as kudu/127.26.120.254@KRBTEST.COM (short username kudu)
I20260929 10:59:25.456203 30022 webserver.cc:460] Webserver started at http://127.26.120.254:41951/ using document root <none> and password file <none>
I20260929 10:59:25.456447 30022 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:25.456513 30022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:25.456661 30022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:25.457705 30022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "843883d8e6d74a86bd7e27ea99c264c4"
format_stamp: "Formatted at 2026-09-29 10:59:25 on dist-test-slave-2lzq"
server_key: "f9418b31ec8de3e970fbda05b2314a16"
server_key_iv: "df8b81ff46071c1ef4e4ae58a74eb400"
server_key_version: "encryptionkey@0"
I20260929 10:59:25.458109 30022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "843883d8e6d74a86bd7e27ea99c264c4"
format_stamp: "Formatted at 2026-09-29 10:59:25 on dist-test-slave-2lzq"
server_key: "f9418b31ec8de3e970fbda05b2314a16"
server_key_iv: "df8b81ff46071c1ef4e4ae58a74eb400"
server_key_version: "encryptionkey@0"
I20260929 10:59:25.459407 30022 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:59:25.460124 30037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:25.460296 30022 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:59:25.460397 30022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "843883d8e6d74a86bd7e27ea99c264c4"
format_stamp: "Formatted at 2026-09-29 10:59:25 on dist-test-slave-2lzq"
server_key: "f9418b31ec8de3e970fbda05b2314a16"
server_key_iv: "df8b81ff46071c1ef4e4ae58a74eb400"
server_key_version: "encryptionkey@0"
I20260929 10:59:25.460477 30022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:25.485148 30022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:25.487242 30022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:25.487419 30022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:25.491526 30022 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:34291
I20260929 10:59:25.491578 30089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:34291 every 8 connection(s)
I20260929 10:59:25.492151 30022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 10:59:25.492796 30090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:25.495103 30090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 843883d8e6d74a86bd7e27ea99c264c4: Bootstrap starting.
I20260929 10:59:25.495620 30090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 843883d8e6d74a86bd7e27ea99c264c4: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:25.495853 30090 log.cc:824] T 00000000000000000000000000000000 P 843883d8e6d74a86bd7e27ea99c264c4: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:25.496456 30090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 843883d8e6d74a86bd7e27ea99c264c4: No bootstrap required, opened a new log
I20260929 10:59:25.497845 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 30022
I20260929 10:59:25.497920 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 10:59:25.498148 27107 external_mini_cluster.cc:1428] Setting key d36ba11bc6a7c9c35ad1f02f981b603c
I20260929 10:59:25.498128 30090 raft_consensus.cc:348] T 00000000000000000000000000000000 P 843883d8e6d74a86bd7e27ea99c264c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843883d8e6d74a86bd7e27ea99c264c4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34291 } }
I20260929 10:59:25.498334 30090 raft_consensus.cc:374] T 00000000000000000000000000000000 P 843883d8e6d74a86bd7e27ea99c264c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:25.498383 30090 raft_consensus.cc:729] T 00000000000000000000000000000000 P 843883d8e6d74a86bd7e27ea99c264c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843883d8e6d74a86bd7e27ea99c264c4, State: Initialized, Role: FOLLOWER
I20260929 10:59:25.498526 30090 consensus_queue.cc:260] T 00000000000000000000000000000000 P 843883d8e6d74a86bd7e27ea99c264c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843883d8e6d74a86bd7e27ea99c264c4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34291 } }
I20260929 10:59:25.498617 30090 raft_consensus.cc:388] T 00000000000000000000000000000000 P 843883d8e6d74a86bd7e27ea99c264c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:59:25.498674 30090 raft_consensus.cc:482] T 00000000000000000000000000000000 P 843883d8e6d74a86bd7e27ea99c264c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:59:25.498730 30090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 843883d8e6d74a86bd7e27ea99c264c4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:25.499337 30090 raft_consensus.cc:504] T 00000000000000000000000000000000 P 843883d8e6d74a86bd7e27ea99c264c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843883d8e6d74a86bd7e27ea99c264c4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34291 } }
I20260929 10:59:25.499452 30090 leader_election.cc:302] T 00000000000000000000000000000000 P 843883d8e6d74a86bd7e27ea99c264c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 843883d8e6d74a86bd7e27ea99c264c4; no voters:
I20260929 10:59:25.499606 30090 leader_election.cc:288] T 00000000000000000000000000000000 P 843883d8e6d74a86bd7e27ea99c264c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:59:25.499658 30095 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 843883d8e6d74a86bd7e27ea99c264c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:59:25.499876 30095 raft_consensus.cc:686] T 00000000000000000000000000000000 P 843883d8e6d74a86bd7e27ea99c264c4 [term 1 LEADER]: Becoming Leader. State: Replica: 843883d8e6d74a86bd7e27ea99c264c4, State: Running, Role: LEADER
I20260929 10:59:25.499941 30090 sys_catalog.cc:564] T 00000000000000000000000000000000 P 843883d8e6d74a86bd7e27ea99c264c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:59:25.500028 30095 consensus_queue.cc:237] T 00000000000000000000000000000000 P 843883d8e6d74a86bd7e27ea99c264c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843883d8e6d74a86bd7e27ea99c264c4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34291 } }
I20260929 10:59:25.500474 30095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 843883d8e6d74a86bd7e27ea99c264c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 843883d8e6d74a86bd7e27ea99c264c4. Latest consensus state: current_term: 1 leader_uuid: "843883d8e6d74a86bd7e27ea99c264c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843883d8e6d74a86bd7e27ea99c264c4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34291 } } }
I20260929 10:59:25.500504 30096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 843883d8e6d74a86bd7e27ea99c264c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "843883d8e6d74a86bd7e27ea99c264c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843883d8e6d74a86bd7e27ea99c264c4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34291 } } }
I20260929 10:59:25.500664 30096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 843883d8e6d74a86bd7e27ea99c264c4 [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:25.500866 30095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 843883d8e6d74a86bd7e27ea99c264c4 [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:25.500885 30104 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:59:25.501451 30104 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 29 10:59:25 dist-test-slave-2lzq krb5kdc[30009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:25.507361 30104 catalog_manager.cc:1347] Generated new cluster ID: 69421ee5bdbc4d2cbabcb276a4ace34d
I20260929 10:59:25.507417 30104 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:59:25.509850 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:25.498623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57302 (local address 127.26.120.254:34291)
0929 10:59:25.499242 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:25.499252 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:25.499287 (+ 35us) server_negotiation.cc:407] Connection header received
0929 10:59:25.499405 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:25.499413 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:25.499473 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:25.499643 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:25.500508 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:25.501910 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:25.502834 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:25.503465 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:25.503595 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:25.506009 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:25.506013 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:25.506014 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:25.506026 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:25.508308 (+ 2282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:25.508678 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:25.508681 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:25.508681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:25.508786 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:25.508986 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:25.508987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:25.508987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:25.509230 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:25.509393 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:25.509488 (+ 95us) server_negotiation.cc:299] Negotiation successful
0929 10:59:25.509638 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":503,"threads_started":1}
I20260929 10:59:25.514782 27107 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:59:25.530367 30104 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:59:25.530594 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:25.527020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57304 (local address 127.26.120.254:34291)
0929 10:59:25.527069 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:25.527071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:25.527210 (+ 139us) server_negotiation.cc:407] Connection header received
0929 10:59:25.527292 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:25.527293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:25.527306 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:25.527378 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:25.527767 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:25.528264 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:25.529005 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:25.529226 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:25.529318 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:25.530038 (+ 720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:25.530040 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:25.530040 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:59:25.530306 (+ 266us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:25.530326 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:59:25.530333 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:25.530396 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:25.530502 (+ 106us) server_negotiation.cc:299] Negotiation successful
0929 10:59:25.530528 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 10:59:25.531111 30104 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:59:25.534078 30104 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 843883d8e6d74a86bd7e27ea99c264c4: Generated new TSK 0
I20260929 10:59:25.534269 30104 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:59:25.536544 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 30022
2026-09-29T10:59:25Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1304 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:59:25 dist-test-slave-2lzq krb5kdc[30123](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:59:25 dist-test-slave-2lzq krb5kdc[30123](info): set up 2 sockets
Sep 29 10:59:25 dist-test-slave-2lzq krb5kdc[30123](info): commencing operation
krb5kdc: starting...
W20260929 10:59:26.570250 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 10:59:26 dist-test-slave-2lzq krb5kdc[30123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:59:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:59:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.120.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
I20260929 10:59:26.684122 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:37903
--webserver_interface=127.26.120.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:39033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:37903
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:26.766471 30136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:26.766804 30136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:26.766845 30136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:26.768325 30136 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:59:26.768368 30136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:26.768383 30136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:26.768396 30136 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:59:26.768409 30136 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:59:26.769804 30136 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:39033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:37903
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:37903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:26.769989 30136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:26.770221 30136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:26.774821 30144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:26.774814 30142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:26.774940 30136 server_base.cc:1034] running on GCE node
W20260929 10:59:26.775049 30141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:26.775357 30136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:26.775655 30136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:26.776818 30136 hybrid_clock.cc:648] HybridClock initialized: now 1790679566776815 us; error 36 us; skew 500 ppm
Sep 29 10:59:26 dist-test-slave-2lzq krb5kdc[30123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679566, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:26.779385 30136 init.cc:377] Logged in from keytab as kudu/127.26.120.254@KRBTEST.COM (short username kudu)
I20260929 10:59:26.779881 30136 webserver.cc:460] Webserver started at http://127.26.120.254:44471/ using document root <none> and password file <none>
I20260929 10:59:26.780113 30136 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:26.780182 30136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:26.780336 30136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:26.781374 30136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "0c944cd2378f4a2bb420969a93135b34"
format_stamp: "Formatted at 2026-09-29 10:59:26 on dist-test-slave-2lzq"
server_key: "89009c58414c9c83de58f07a05dfbe46"
server_key_iv: "85215b316112a31d2a21eb7396e5260f"
server_key_version: "encryptionkey@0"
I20260929 10:59:26.781757 30136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "0c944cd2378f4a2bb420969a93135b34"
format_stamp: "Formatted at 2026-09-29 10:59:26 on dist-test-slave-2lzq"
server_key: "89009c58414c9c83de58f07a05dfbe46"
server_key_iv: "85215b316112a31d2a21eb7396e5260f"
server_key_version: "encryptionkey@0"
I20260929 10:59:26.783160 30136 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:59:26.783864 30151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:26.784042 30136 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:59:26.784150 30136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "0c944cd2378f4a2bb420969a93135b34"
format_stamp: "Formatted at 2026-09-29 10:59:26 on dist-test-slave-2lzq"
server_key: "89009c58414c9c83de58f07a05dfbe46"
server_key_iv: "85215b316112a31d2a21eb7396e5260f"
server_key_version: "encryptionkey@0"
I20260929 10:59:26.784227 30136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:26.791575 30136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:26.793854 30136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:26.794015 30136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:26.797792 30136 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:37903
I20260929 10:59:26.797855 30203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:37903 every 8 connection(s)
I20260929 10:59:26.798306 30136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 10:59:26.798755 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 30136
I20260929 10:59:26.798864 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 10:59:26.799038 27107 external_mini_cluster.cc:1428] Setting key a32ab6726b66b6a9f472da502ff5946c
I20260929 10:59:26.799525 30204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:26.802760 30204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c944cd2378f4a2bb420969a93135b34: Bootstrap starting.
I20260929 10:59:26.803382 30204 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c944cd2378f4a2bb420969a93135b34: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:26.803686 30204 log.cc:824] T 00000000000000000000000000000000 P 0c944cd2378f4a2bb420969a93135b34: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:26.804399 30204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c944cd2378f4a2bb420969a93135b34: No bootstrap required, opened a new log
Sep 29 10:59:26 dist-test-slave-2lzq krb5kdc[30123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:26.806420 30204 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c944cd2378f4a2bb420969a93135b34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c944cd2378f4a2bb420969a93135b34" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 37903 } }
I20260929 10:59:26.806598 30204 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c944cd2378f4a2bb420969a93135b34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:26.806648 30204 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c944cd2378f4a2bb420969a93135b34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c944cd2378f4a2bb420969a93135b34, State: Initialized, Role: FOLLOWER
I20260929 10:59:26.806787 30204 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c944cd2378f4a2bb420969a93135b34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c944cd2378f4a2bb420969a93135b34" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 37903 } }
I20260929 10:59:26.806892 30204 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c944cd2378f4a2bb420969a93135b34 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:59:26.806973 30204 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c944cd2378f4a2bb420969a93135b34 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:59:26.807066 30204 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c944cd2378f4a2bb420969a93135b34 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:26.807753 30204 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c944cd2378f4a2bb420969a93135b34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c944cd2378f4a2bb420969a93135b34" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 37903 } }
I20260929 10:59:26.807880 30204 leader_election.cc:302] T 00000000000000000000000000000000 P 0c944cd2378f4a2bb420969a93135b34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c944cd2378f4a2bb420969a93135b34; no voters:
I20260929 10:59:26.808100 30204 leader_election.cc:288] T 00000000000000000000000000000000 P 0c944cd2378f4a2bb420969a93135b34 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:59:26.808132 30210 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c944cd2378f4a2bb420969a93135b34 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:59:26.808259 30210 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c944cd2378f4a2bb420969a93135b34 [term 1 LEADER]: Becoming Leader. State: Replica: 0c944cd2378f4a2bb420969a93135b34, State: Running, Role: LEADER
I20260929 10:59:26.808396 30204 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c944cd2378f4a2bb420969a93135b34 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:59:26.808420 30210 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c944cd2378f4a2bb420969a93135b34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c944cd2378f4a2bb420969a93135b34" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 37903 } }
I20260929 10:59:26.808784 30211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c944cd2378f4a2bb420969a93135b34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c944cd2378f4a2bb420969a93135b34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c944cd2378f4a2bb420969a93135b34" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 37903 } } }
I20260929 10:59:26.808943 30211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c944cd2378f4a2bb420969a93135b34 [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:26.808771 30210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c944cd2378f4a2bb420969a93135b34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c944cd2378f4a2bb420969a93135b34. Latest consensus state: current_term: 1 leader_uuid: "0c944cd2378f4a2bb420969a93135b34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c944cd2378f4a2bb420969a93135b34" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 37903 } } }
I20260929 10:59:26.809239 30210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c944cd2378f4a2bb420969a93135b34 [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:26.809518 30219 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:59:26.810081 30219 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:59:26.810472 30208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:26.799586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49890 (local address 127.26.120.254:37903)
0929 10:59:26.799772 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:26.799781 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:26.799812 (+ 31us) server_negotiation.cc:407] Connection header received
0929 10:59:26.800345 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:26.800353 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:26.800430 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:26.800589 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:26.801147 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:26.802538 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:26.803326 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:26.803946 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:26.804073 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:26.806385 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:26.806389 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:26.806390 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:26.806402 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:26.808327 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:26.808740 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:26.808743 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:26.808744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:26.808850 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:26.809102 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:26.809103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:26.809103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:26.809408 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:26.810080 (+ 672us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:26.810153 (+ 73us) server_negotiation.cc:299] Negotiation successful
0929 10:59:26.810285 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":87,"threads_started":1}
I20260929 10:59:26.811579 30219 catalog_manager.cc:1347] Generated new cluster ID: b8fd6e757baa4bfcad3f06019d25a5fe
I20260929 10:59:26.811630 30219 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:59:26.818427 30219 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:59:26.818931 30219 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:59:26.825138 30219 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c944cd2378f4a2bb420969a93135b34: Generated new TSK 0
I20260929 10:59:26.825312 30219 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.120.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab.
Entry for principal kudu/127.26.120.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab.
I20260929 10:59:26.856172 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.193:0
--local_ip_for_outbound_sockets=127.26.120.193
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:37903
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:39033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:26.935724 30231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:26.935925 30231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:26.935948 30231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:26.937016 30231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:26.937050 30231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:26.937076 30231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.193
I20260929 10:59:26.938500 30231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:39033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:37903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.193
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:26.938690 30231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:26.938864 30231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:26.943756 30236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:26.943795 30237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:26.943833 30231 server_base.cc:1034] running on GCE node
W20260929 10:59:26.943735 30239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:26.944075 30231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:26.944262 30231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:26.945398 30231 hybrid_clock.cc:648] HybridClock initialized: now 1790679566945383 us; error 32 us; skew 500 ppm
Sep 29 10:59:26 dist-test-slave-2lzq krb5kdc[30123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679566, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:26.947772 30231 init.cc:377] Logged in from keytab as kudu/127.26.120.193@KRBTEST.COM (short username kudu)
I20260929 10:59:26.948189 30231 webserver.cc:460] Webserver started at http://127.26.120.193:44209/ using document root <none> and password file <none>
I20260929 10:59:26.948418 30231 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:26.948465 30231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:26.948633 30231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:26.949652 30231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-0/data/instance:
uuid: "d2055b387b2f4ba9bf534679b9693adc"
format_stamp: "Formatted at 2026-09-29 10:59:26 on dist-test-slave-2lzq"
server_key: "8cc3d313adc0be0a72ed93a1a49f18cc"
server_key_iv: "d9e2e3ced4155b66b8d5c6e8ef1bb50f"
server_key_version: "encryptionkey@0"
I20260929 10:59:26.950007 30231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance:
uuid: "d2055b387b2f4ba9bf534679b9693adc"
format_stamp: "Formatted at 2026-09-29 10:59:26 on dist-test-slave-2lzq"
server_key: "8cc3d313adc0be0a72ed93a1a49f18cc"
server_key_iv: "d9e2e3ced4155b66b8d5c6e8ef1bb50f"
server_key_version: "encryptionkey@0"
I20260929 10:59:26.951316 30231 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:59:26.951959 30246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:26.952138 30231 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:59:26.952224 30231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-0/wal
uuid: "d2055b387b2f4ba9bf534679b9693adc"
format_stamp: "Formatted at 2026-09-29 10:59:26 on dist-test-slave-2lzq"
server_key: "8cc3d313adc0be0a72ed93a1a49f18cc"
server_key_iv: "d9e2e3ced4155b66b8d5c6e8ef1bb50f"
server_key_version: "encryptionkey@0"
I20260929 10:59:26.952297 30231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:26.963328 30231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:26.965008 30231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:26.965157 30231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:26.965361 30231 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:59:26.965647 30231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:26.965680 30231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:26.965703 30231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:26.965746 30231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:26.971637 30231 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.193:38057
I20260929 10:59:26.971683 30359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.193:38057 every 8 connection(s)
I20260929 10:59:26.971969 30231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
Sep 29 10:59:26 dist-test-slave-2lzq krb5kdc[30123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679566, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:26.980466 30208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:26.972643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:51451 (local address 127.26.120.254:37903)
0929 10:59:26.972721 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:26.972723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:26.972909 (+ 186us) server_negotiation.cc:407] Connection header received
0929 10:59:26.973354 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:26.973354 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:26.973364 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:26.973423 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:26.974338 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:26.974863 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:26.975791 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:26.975979 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:26.976029 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:26.977938 (+ 1909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:26.977939 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:26.977939 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:26.977946 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:26.979356 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:26.979691 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:26.979692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:26.979692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:26.979742 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:26.979938 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:26.979939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:26.979939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:26.980034 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:26.980089 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:26.980344 (+ 255us) server_negotiation.cc:299] Negotiation successful
0929 10:59:26.980403 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 10:59:26.980610 30362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:26.972588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:37903 (local address 127.26.120.193:51451)
0929 10:59:26.972785 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:26.972831 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:26.973273 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:26.973476 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:26.973479 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:26.974017 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:26.974259 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:26.974263 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:26.974938 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:26.974938 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:26.975715 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:26.975717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:26.976047 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:26.976048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:26.976141 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:26.976543 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:26.976550 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:26.977856 (+ 1306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:26.979423 (+ 1567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:26.979428 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:26.979430 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:26.979627 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:26.979797 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:26.979797 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:26.979798 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:26.979886 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:26.980086 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:26.980089 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:26.980233 (+ 144us) client_negotiation.cc:769] Sending connection context
0929 10:59:26.980315 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 10:59:26.980424 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":87,"threads_started":1}
I20260929 10:59:26.980798 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 30231
I20260929 10:59:26.980927 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance
I20260929 10:59:26.981155 27107 external_mini_cluster.cc:1428] Setting key a6e9f93987ea942058c7b98b8eb532e6
I20260929 10:59:26.981163 30360 heartbeater.cc:347] Connected to a master server at 127.26.120.254:37903
I20260929 10:59:26.981251 30360 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:26.981547 30360 heartbeater.cc:511] Master 127.26.120.254:37903 requested a full tablet report, sending...
I20260929 10:59:26.982138 30168 ts_manager.cc:194] Registered new tserver with Master: d2055b387b2f4ba9bf534679b9693adc (127.26.120.193:38057)
I20260929 10:59:26.983016 30168 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.193@KRBTEST.COM'} at 127.26.120.193:51451
WARNING: no policy specified for kudu/127.26.120.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab.
Entry for principal kudu/127.26.120.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab.
I20260929 10:59:27.005143 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.194:0
--local_ip_for_outbound_sockets=127.26.120.194
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:37903
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:39033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:27.085170 30365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:27.085413 30365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:27.085461 30365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:27.086700 30365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:27.086762 30365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:27.086807 30365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.194
I20260929 10:59:27.088375 30365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:39033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:37903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.194
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:27.088647 30365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:27.088877 30365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:27.093745 30371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:27.093735 30373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:27.093735 30370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:27.093941 30365 server_base.cc:1034] running on GCE node
I20260929 10:59:27.094165 30365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:27.094411 30365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:27.095572 30365 hybrid_clock.cc:648] HybridClock initialized: now 1790679567095570 us; error 38 us; skew 500 ppm
Sep 29 10:59:27 dist-test-slave-2lzq krb5kdc[30123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679567, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:27.098253 30365 init.cc:377] Logged in from keytab as kudu/127.26.120.194@KRBTEST.COM (short username kudu)
I20260929 10:59:27.098666 30365 webserver.cc:460] Webserver started at http://127.26.120.194:42361/ using document root <none> and password file <none>
I20260929 10:59:27.098891 30365 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:27.098956 30365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:27.099089 30365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:27.100216 30365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-1/data/instance:
uuid: "0f3c99416d09447fb54d67265d9e2e2e"
format_stamp: "Formatted at 2026-09-29 10:59:27 on dist-test-slave-2lzq"
server_key: "979f587a89e25aefcf3e5b4d396dfe55"
server_key_iv: "0905e2a4fb1e22c8e4c6896ce7d2e239"
server_key_version: "encryptionkey@0"
I20260929 10:59:27.100586 30365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance:
uuid: "0f3c99416d09447fb54d67265d9e2e2e"
format_stamp: "Formatted at 2026-09-29 10:59:27 on dist-test-slave-2lzq"
server_key: "979f587a89e25aefcf3e5b4d396dfe55"
server_key_iv: "0905e2a4fb1e22c8e4c6896ce7d2e239"
server_key_version: "encryptionkey@0"
I20260929 10:59:27.101928 30365 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 10:59:27.102700 30380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:27.102867 30365 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:27.102964 30365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-1/wal
uuid: "0f3c99416d09447fb54d67265d9e2e2e"
format_stamp: "Formatted at 2026-09-29 10:59:27 on dist-test-slave-2lzq"
server_key: "979f587a89e25aefcf3e5b4d396dfe55"
server_key_iv: "0905e2a4fb1e22c8e4c6896ce7d2e239"
server_key_version: "encryptionkey@0"
I20260929 10:59:27.103045 30365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:27.131011 30365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:27.132884 30365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:27.133046 30365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:27.133287 30365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:59:27.133602 30365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:27.133656 30365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:27.133692 30365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:27.133720 30365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:27.139370 30365 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.194:45963
I20260929 10:59:27.139415 30493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.194:45963 every 8 connection(s)
I20260929 10:59:27.139724 30365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
I20260929 10:59:27.140276 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 30365
I20260929 10:59:27.140393 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance
I20260929 10:59:27.140594 27107 external_mini_cluster.cc:1428] Setting key bdb57250a3c870c5e51471671347d47f
Sep 29 10:59:27 dist-test-slave-2lzq krb5kdc[30123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679567, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.120.195@KRBTEST.COM; defaulting to no policy
I20260929 10:59:27.150019 30208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:27.140545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:54893 (local address 127.26.120.254:37903)
0929 10:59:27.140624 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:27.140626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:27.140942 (+ 316us) server_negotiation.cc:407] Connection header received
0929 10:59:27.141388 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:27.141389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:27.141401 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:27.141476 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:27.142468 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:27.143059 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:27.144607 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:27.144878 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:27.144945 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:27.147300 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:27.147301 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:27.147302 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:27.147309 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:27.148746 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:27.149210 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:27.149211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:27.149212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:27.149263 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:27.149468 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:27.149468 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:27.149469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:27.149576 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:27.149627 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:27.149883 (+ 256us) server_negotiation.cc:299] Negotiation successful
0929 10:59:27.149955 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 10:59:27.150242 30496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:27.140530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:37903 (local address 127.26.120.194:54893)
0929 10:59:27.140718 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:27.140761 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:27.141287 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:27.141535 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:27.141538 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:27.142184 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:27.142372 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:27.142377 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:27.143150 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:27.143151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:27.144488 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:27.144492 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:27.144976 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:27.144977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:27.145131 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:27.145598 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:27.145609 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:27.147200 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:27.148844 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:27.148851 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:27.148854 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:27.149145 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:27.149322 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:27.149323 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:27.149323 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:27.149415 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:27.149638 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:27.149642 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:27.149788 (+ 146us) client_negotiation.cc:769] Sending connection context
0929 10:59:27.149870 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 10:59:27.149979 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":80,"threads_started":1}
I20260929 10:59:27.150796 30494 heartbeater.cc:347] Connected to a master server at 127.26.120.254:37903
I20260929 10:59:27.150925 30494 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:27.151140 30494 heartbeater.cc:511] Master 127.26.120.254:37903 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.195@KRBTEST.COM" created.
I20260929 10:59:27.151538 30168 ts_manager.cc:194] Registered new tserver with Master: 0f3c99416d09447fb54d67265d9e2e2e (127.26.120.194:45963)
I20260929 10:59:27.152073 30168 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.194@KRBTEST.COM'} at 127.26.120.194:54893
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab.
Entry for principal kudu/127.26.120.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab.
I20260929 10:59:27.171088 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.195:0
--local_ip_for_outbound_sockets=127.26.120.195
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:37903
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:39033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:27.263374 30499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:27.263564 30499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:27.263586 30499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:27.264679 30499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:27.264719 30499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:27.264744 30499 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.195
I20260929 10:59:27.266191 30499 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:39033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:37903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.195
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:27.266429 30499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:27.266661 30499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:27.271560 30507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:27.271673 30505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:27.271595 30504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:27.271812 30499 server_base.cc:1034] running on GCE node
I20260929 10:59:27.272074 30499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:27.272270 30499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:27.273418 30499 hybrid_clock.cc:648] HybridClock initialized: now 1790679567273407 us; error 28 us; skew 500 ppm
Sep 29 10:59:27 dist-test-slave-2lzq krb5kdc[30123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679567, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:27.275817 30499 init.cc:377] Logged in from keytab as kudu/127.26.120.195@KRBTEST.COM (short username kudu)
I20260929 10:59:27.276204 30499 webserver.cc:460] Webserver started at http://127.26.120.195:39191/ using document root <none> and password file <none>
I20260929 10:59:27.276428 30499 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:27.276472 30499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:27.276566 30499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:27.277544 30499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-2/data/instance:
uuid: "5057fc0c05e84919a9f0baec2f912df4"
format_stamp: "Formatted at 2026-09-29 10:59:27 on dist-test-slave-2lzq"
server_key: "7462b958de5a7d28177adfd8afb94a1d"
server_key_iv: "5003b3dd265820457b2435673408b8a6"
server_key_version: "encryptionkey@0"
I20260929 10:59:27.277834 30499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance:
uuid: "5057fc0c05e84919a9f0baec2f912df4"
format_stamp: "Formatted at 2026-09-29 10:59:27 on dist-test-slave-2lzq"
server_key: "7462b958de5a7d28177adfd8afb94a1d"
server_key_iv: "5003b3dd265820457b2435673408b8a6"
server_key_version: "encryptionkey@0"
I20260929 10:59:27.279160 30499 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:59:27.279834 30514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:27.279999 30499 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:59:27.280097 30499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-2/wal
uuid: "5057fc0c05e84919a9f0baec2f912df4"
format_stamp: "Formatted at 2026-09-29 10:59:27 on dist-test-slave-2lzq"
server_key: "7462b958de5a7d28177adfd8afb94a1d"
server_key_iv: "5003b3dd265820457b2435673408b8a6"
server_key_version: "encryptionkey@0"
I20260929 10:59:27.280177 30499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:27.310995 30499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:27.312923 30499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:27.313092 30499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:27.313329 30499 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:59:27.313640 30499 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:27.313694 30499 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:27.313735 30499 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:27.313763 30499 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:27.319792 30499 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.195:42911
I20260929 10:59:27.319835 30627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.195:42911 every 8 connection(s)
I20260929 10:59:27.320140 30499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
Sep 29 10:59:27 dist-test-slave-2lzq krb5kdc[30123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679567, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:27.326941 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 30499
I20260929 10:59:27.327054 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance
I20260929 10:59:27.327234 27107 external_mini_cluster.cc:1428] Setting key 5e489372f47057023d50f5f285936037
I20260929 10:59:27.329752 30208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:27.320851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:39331 (local address 127.26.120.254:37903)
0929 10:59:27.320895 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:27.320897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:27.321113 (+ 216us) server_negotiation.cc:407] Connection header received
0929 10:59:27.321684 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:27.321685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:27.321699 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:27.321754 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:27.322758 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:27.323277 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:27.324298 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:27.324491 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:27.324566 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:27.326776 (+ 2210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:27.326778 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:27.326778 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:27.326786 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:27.328387 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:27.328792 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:27.328793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:27.328794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:27.328851 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:27.329119 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:27.329120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:27.329120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:27.329238 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:27.329308 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:27.329581 (+ 273us) server_negotiation.cc:299] Negotiation successful
0929 10:59:27.329662 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 10:59:27.329937 30630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:27.320826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:37903 (local address 127.26.120.195:39331)
0929 10:59:27.320997 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:27.321032 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:27.321568 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:27.321832 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:27.321836 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:27.322470 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:27.322657 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:27.322662 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:27.323383 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:27.323384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:27.324197 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:27.324198 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:27.324572 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:27.324573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:27.324698 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:27.325169 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:27.325179 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:27.326681 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:27.328475 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:27.328481 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:27.328483 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:27.328715 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:27.328926 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:27.328927 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:27.328928 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:27.329041 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:27.329312 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:27.329318 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:27.329449 (+ 131us) client_negotiation.cc:769] Sending connection context
0929 10:59:27.329561 (+ 112us) client_negotiation.cc:240] Negotiation successful
0929 10:59:27.329715 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":83,"threads_started":1}
I20260929 10:59:27.330381 30628 heartbeater.cc:347] Connected to a master server at 127.26.120.254:37903
I20260929 10:59:27.330458 30628 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:27.330655 30628 heartbeater.cc:511] Master 127.26.120.254:37903 requested a full tablet report, sending...
I20260929 10:59:27.330978 30168 ts_manager.cc:194] Registered new tserver with Master: 5057fc0c05e84919a9f0baec2f912df4 (127.26.120.195:42911)
I20260929 10:59:27.331326 30168 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.195@KRBTEST.COM'} at 127.26.120.195:39331
I20260929 10:59:27.338833 27107 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 10:59:27 dist-test-slave-2lzq krb5kdc[30123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679567, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 29 10:59:27 dist-test-slave-2lzq krb5kdc[30123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679567, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:27.360291 30208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:27.353584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49906 (local address 127.26.120.254:37903)
0929 10:59:27.353649 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:27.353651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:27.353722 (+ 71us) server_negotiation.cc:407] Connection header received
0929 10:59:27.353765 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:27.353766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:27.353775 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:27.353828 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:27.354454 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:27.354941 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:27.355727 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:27.355911 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:27.355963 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:27.357980 (+ 2017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:27.357981 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:27.357982 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:27.357988 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:27.359261 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:27.359618 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:27.359618 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:27.359619 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:27.359668 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:27.359842 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:27.359843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:27.359843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:27.359983 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:27.360058 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:27.360172 (+ 114us) server_negotiation.cc:299] Negotiation successful
0929 10:59:27.360228 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260929 10:59:27.360590 30168 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:49906
I20260929 10:59:27.361039 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 30231
I20260929 10:59:27.365543 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 30365
I20260929 10:59:27.371037 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 30499
I20260929 10:59:27.375883 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 30136
2026-09-29T10:59:27Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1841 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:59:27 dist-test-slave-2lzq krb5kdc[30640](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:59:27 dist-test-slave-2lzq krb5kdc[30640](info): set up 2 sockets
Sep 29 10:59:27 dist-test-slave-2lzq krb5kdc[30640](info): commencing operation
krb5kdc: starting...
W20260929 10:59:28.406220 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 10:59:28 dist-test-slave-2lzq krb5kdc[30640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:59:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:59:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.120.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
I20260929 10:59:28.521363 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:44111
--webserver_interface=127.26.120.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:43079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:44111
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:28.601006 30653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:28.601260 30653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:28.601300 30653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:28.602495 30653 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:59:28.602563 30653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:28.602587 30653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:28.602614 30653 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:59:28.602646 30653 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:59:28.604131 30653 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:43079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:44111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:44111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:28.604411 30653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:28.604671 30653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:28.609615 30658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:28.609583 30661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:28.609745 30653 server_base.cc:1034] running on GCE node
W20260929 10:59:28.609583 30659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:28.610186 30653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:28.610486 30653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:28.611636 30653 hybrid_clock.cc:648] HybridClock initialized: now 1790679568611616 us; error 37 us; skew 500 ppm
Sep 29 10:59:28 dist-test-slave-2lzq krb5kdc[30640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679568, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:28.614251 30653 init.cc:377] Logged in from keytab as kudu/127.26.120.254@KRBTEST.COM (short username kudu)
I20260929 10:59:28.614696 30653 webserver.cc:460] Webserver started at http://127.26.120.254:36981/ using document root <none> and password file <none>
I20260929 10:59:28.614877 30653 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:28.614920 30653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:28.615007 30653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:28.615985 30653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "b8cdf24cb4374ce99e44d6dd730d5aa8"
format_stamp: "Formatted at 2026-09-29 10:59:28 on dist-test-slave-2lzq"
server_key: "669a79dfbf4e2cb99249c25229915f07"
server_key_iv: "e59273c8c905f55006fd06585963d4b3"
server_key_version: "encryptionkey@0"
I20260929 10:59:28.616291 30653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "b8cdf24cb4374ce99e44d6dd730d5aa8"
format_stamp: "Formatted at 2026-09-29 10:59:28 on dist-test-slave-2lzq"
server_key: "669a79dfbf4e2cb99249c25229915f07"
server_key_iv: "e59273c8c905f55006fd06585963d4b3"
server_key_version: "encryptionkey@0"
I20260929 10:59:28.617605 30653 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:59:28.618351 30668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:28.618579 30653 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:59:28.618687 30653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "b8cdf24cb4374ce99e44d6dd730d5aa8"
format_stamp: "Formatted at 2026-09-29 10:59:28 on dist-test-slave-2lzq"
server_key: "669a79dfbf4e2cb99249c25229915f07"
server_key_iv: "e59273c8c905f55006fd06585963d4b3"
server_key_version: "encryptionkey@0"
I20260929 10:59:28.618765 30653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:28.628961 30653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:28.631275 30653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:28.631436 30653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:28.635684 30653 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:44111
I20260929 10:59:28.635715 30720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:44111 every 8 connection(s)
I20260929 10:59:28.636238 30653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 10:59:28.637002 30721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:28.639348 30721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8cdf24cb4374ce99e44d6dd730d5aa8: Bootstrap starting.
I20260929 10:59:28.639936 30721 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8cdf24cb4374ce99e44d6dd730d5aa8: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:28.640218 30721 log.cc:824] T 00000000000000000000000000000000 P b8cdf24cb4374ce99e44d6dd730d5aa8: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:28.640940 30721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8cdf24cb4374ce99e44d6dd730d5aa8: No bootstrap required, opened a new log
I20260929 10:59:28.642666 30721 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8cdf24cb4374ce99e44d6dd730d5aa8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8cdf24cb4374ce99e44d6dd730d5aa8" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44111 } }
I20260929 10:59:28.642829 30721 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8cdf24cb4374ce99e44d6dd730d5aa8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:28.642870 30721 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8cdf24cb4374ce99e44d6dd730d5aa8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8cdf24cb4374ce99e44d6dd730d5aa8, State: Initialized, Role: FOLLOWER
I20260929 10:59:28.643038 30721 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8cdf24cb4374ce99e44d6dd730d5aa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8cdf24cb4374ce99e44d6dd730d5aa8" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44111 } }
I20260929 10:59:28.643124 30721 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8cdf24cb4374ce99e44d6dd730d5aa8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:59:28.643163 30721 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8cdf24cb4374ce99e44d6dd730d5aa8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:59:28.643213 30721 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8cdf24cb4374ce99e44d6dd730d5aa8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:28.644119 30721 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8cdf24cb4374ce99e44d6dd730d5aa8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8cdf24cb4374ce99e44d6dd730d5aa8" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44111 } }
I20260929 10:59:28.644239 30721 leader_election.cc:302] T 00000000000000000000000000000000 P b8cdf24cb4374ce99e44d6dd730d5aa8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8cdf24cb4374ce99e44d6dd730d5aa8; no voters:
I20260929 10:59:28.644438 30721 leader_election.cc:288] T 00000000000000000000000000000000 P b8cdf24cb4374ce99e44d6dd730d5aa8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:59:28.644503 30724 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8cdf24cb4374ce99e44d6dd730d5aa8 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:59:28.644702 30724 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8cdf24cb4374ce99e44d6dd730d5aa8 [term 1 LEADER]: Becoming Leader. State: Replica: b8cdf24cb4374ce99e44d6dd730d5aa8, State: Running, Role: LEADER
I20260929 10:59:28.644771 30721 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8cdf24cb4374ce99e44d6dd730d5aa8 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:59:28.644784 30724 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8cdf24cb4374ce99e44d6dd730d5aa8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8cdf24cb4374ce99e44d6dd730d5aa8" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44111 } }
I20260929 10:59:28.645180 30725 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8cdf24cb4374ce99e44d6dd730d5aa8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8cdf24cb4374ce99e44d6dd730d5aa8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8cdf24cb4374ce99e44d6dd730d5aa8" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44111 } } }
I20260929 10:59:28.645215 30726 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8cdf24cb4374ce99e44d6dd730d5aa8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8cdf24cb4374ce99e44d6dd730d5aa8. Latest consensus state: current_term: 1 leader_uuid: "b8cdf24cb4374ce99e44d6dd730d5aa8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8cdf24cb4374ce99e44d6dd730d5aa8" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44111 } } }
I20260929 10:59:28.645367 30726 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8cdf24cb4374ce99e44d6dd730d5aa8 [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:28.645622 30725 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8cdf24cb4374ce99e44d6dd730d5aa8 [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:28.645813 30733 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:59:28.646174 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 30653
I20260929 10:59:28.646277 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 10:59:28.646414 30733 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:59:28.646458 27107 external_mini_cluster.cc:1428] Setting key 4cb053f595640693b863e87803bb752d
I20260929 10:59:28.648313 30733 catalog_manager.cc:1347] Generated new cluster ID: 7f6b563cfbef459fa012b751fc77545b
I20260929 10:59:28.648380 30733 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 10:59:28 dist-test-slave-2lzq krb5kdc[30640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:28.656221 30743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:28.646953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54644 (local address 127.26.120.254:44111)
0929 10:59:28.647200 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:28.647211 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:28.647244 (+ 33us) server_negotiation.cc:407] Connection header received
0929 10:59:28.647358 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:28.647368 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:28.647418 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:28.647569 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:28.648267 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:28.649256 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:28.650015 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:28.650489 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:28.650627 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:28.652872 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:28.652876 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:28.652877 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:28.652889 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:28.654623 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:28.655002 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:28.655005 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:28.655005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:28.655106 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:28.655320 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:28.655321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:28.655321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:28.655558 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:28.655730 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:28.655828 (+ 98us) server_negotiation.cc:299] Negotiation successful
0929 10:59:28.656067 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":67,"threads_started":1}
I20260929 10:59:28.658582 30733 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:59:28.659298 30733 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:59:28.667066 30733 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8cdf24cb4374ce99e44d6dd730d5aa8: Generated new TSK 0
I20260929 10:59:28.667237 30733 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.120.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab.
Entry for principal kudu/127.26.120.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab.
I20260929 10:59:28.696388 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.193:0
--local_ip_for_outbound_sockets=127.26.120.193
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:44111
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:43079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:28.780614 30747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:28.780818 30747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:28.780841 30747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:28.782006 30747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:28.782089 30747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:28.782167 30747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.193
I20260929 10:59:28.783735 30747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:43079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:44111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.193
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:28.783977 30747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:28.784221 30747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:28.790398 30752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:28.790416 30755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:28.790416 30753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:28.790740 30747 server_base.cc:1034] running on GCE node
I20260929 10:59:28.790970 30747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:28.791236 30747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:28.792392 30747 hybrid_clock.cc:648] HybridClock initialized: now 1790679568792371 us; error 38 us; skew 500 ppm
Sep 29 10:59:28 dist-test-slave-2lzq krb5kdc[30640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679568, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:28.794988 30747 init.cc:377] Logged in from keytab as kudu/127.26.120.193@KRBTEST.COM (short username kudu)
I20260929 10:59:28.795436 30747 webserver.cc:460] Webserver started at http://127.26.120.193:39369/ using document root <none> and password file <none>
I20260929 10:59:28.795619 30747 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:28.795660 30747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:28.795746 30747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:28.796731 30747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-0/data/instance:
uuid: "eb31cb771ddb4788876d8f1c1f8c2aad"
format_stamp: "Formatted at 2026-09-29 10:59:28 on dist-test-slave-2lzq"
server_key: "5ae56b0b1d60751aa3086ad21c9e3f1d"
server_key_iv: "6d533f629b7ae41761e095a567dabdd7"
server_key_version: "encryptionkey@0"
I20260929 10:59:28.797024 30747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance:
uuid: "eb31cb771ddb4788876d8f1c1f8c2aad"
format_stamp: "Formatted at 2026-09-29 10:59:28 on dist-test-slave-2lzq"
server_key: "5ae56b0b1d60751aa3086ad21c9e3f1d"
server_key_iv: "6d533f629b7ae41761e095a567dabdd7"
server_key_version: "encryptionkey@0"
I20260929 10:59:28.798318 30747 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 10:59:28.799002 30762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:28.799166 30747 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:28.799264 30747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-0/wal
uuid: "eb31cb771ddb4788876d8f1c1f8c2aad"
format_stamp: "Formatted at 2026-09-29 10:59:28 on dist-test-slave-2lzq"
server_key: "5ae56b0b1d60751aa3086ad21c9e3f1d"
server_key_iv: "6d533f629b7ae41761e095a567dabdd7"
server_key_version: "encryptionkey@0"
I20260929 10:59:28.799345 30747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:28.810935 30747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:28.813282 30747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:28.813455 30747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:28.813694 30747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:59:28.814002 30747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:28.814061 30747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:28.814129 30747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:28.814153 30747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:28.820079 30747 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.193:32833
I20260929 10:59:28.820123 30875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.193:32833 every 8 connection(s)
I20260929 10:59:28.820479 30747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
I20260929 10:59:28.822568 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 30747
I20260929 10:59:28.822685 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance
I20260929 10:59:28.822856 27107 external_mini_cluster.cc:1428] Setting key 70cf4121374a5f30892240f836b41537
Sep 29 10:59:28 dist-test-slave-2lzq krb5kdc[30640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679568, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:28.830616 30743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:28.821156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:44273 (local address 127.26.120.254:44111)
0929 10:59:28.821221 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:28.821224 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:28.821420 (+ 196us) server_negotiation.cc:407] Connection header received
0929 10:59:28.822471 (+ 1051us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:28.822473 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:28.822489 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:28.822533 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:28.823339 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:28.823851 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:28.824857 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:28.825094 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:28.825181 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:28.827630 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:28.827632 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:28.827632 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:28.827643 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:28.829206 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:28.829607 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:28.829608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:28.829609 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:28.829670 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:28.829913 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:28.829914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:28.829915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:28.830036 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:28.830151 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:28.830421 (+ 270us) server_negotiation.cc:299] Negotiation successful
0929 10:59:28.830525 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 10:59:28.830793 30878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:28.821108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44111 (local address 127.26.120.193:44273)
0929 10:59:28.821285 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:28.821332 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:28.822362 (+ 1030us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:28.822613 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:28.822617 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:28.822976 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:28.823244 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:28.823256 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:28.823945 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:28.823946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:28.824765 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:28.824767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:28.825191 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:28.825192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:28.825334 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:28.825870 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:28.825877 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:28.827529 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:28.829296 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:28.829303 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:28.829305 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:28.829530 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:28.829737 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:28.829738 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:28.829739 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:28.829846 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:28.830142 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:28.830149 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:28.830291 (+ 142us) client_negotiation.cc:769] Sending connection context
0929 10:59:28.830399 (+ 108us) client_negotiation.cc:240] Negotiation successful
0929 10:59:28.830560 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":92,"threads_started":1}
WARNING: no policy specified for kudu/127.26.120.194@KRBTEST.COM; defaulting to no policy
I20260929 10:59:28.831369 30876 heartbeater.cc:347] Connected to a master server at 127.26.120.254:44111
I20260929 10:59:28.831465 30876 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:28.831672 30876 heartbeater.cc:511] Master 127.26.120.254:44111 requested a full tablet report, sending...
I20260929 10:59:28.832238 30685 ts_manager.cc:194] Registered new tserver with Master: eb31cb771ddb4788876d8f1c1f8c2aad (127.26.120.193:32833)
I20260929 10:59:28.832839 30685 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.193@KRBTEST.COM'} at 127.26.120.193:44273
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab.
Entry for principal kudu/127.26.120.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab.
I20260929 10:59:28.848210 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.194:0
--local_ip_for_outbound_sockets=127.26.120.194
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:44111
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:43079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:28.929270 30881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:28.929512 30881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:28.929549 30881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:28.930778 30881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:28.930845 30881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:28.930891 30881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.194
I20260929 10:59:28.932420 30881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:43079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:44111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.194
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:28.932703 30881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:28.932945 30881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:28.937953 30887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:28.938041 30881 server_base.cc:1034] running on GCE node
W20260929 10:59:28.937947 30890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:28.937943 30888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:28.938390 30881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:28.938593 30881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:28.939765 30881 hybrid_clock.cc:648] HybridClock initialized: now 1790679568939753 us; error 45 us; skew 500 ppm
Sep 29 10:59:28 dist-test-slave-2lzq krb5kdc[30640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679568, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:28.942327 30881 init.cc:377] Logged in from keytab as kudu/127.26.120.194@KRBTEST.COM (short username kudu)
I20260929 10:59:28.942782 30881 webserver.cc:460] Webserver started at http://127.26.120.194:40233/ using document root <none> and password file <none>
I20260929 10:59:28.942976 30881 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:28.943017 30881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:28.943112 30881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:28.944065 30881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-1/data/instance:
uuid: "d4bac528335c4b85b5d162c264194074"
format_stamp: "Formatted at 2026-09-29 10:59:28 on dist-test-slave-2lzq"
server_key: "cc04052a83e5232b8863ab7787113662"
server_key_iv: "61dd33f7372b6e06e476360e8296000d"
server_key_version: "encryptionkey@0"
I20260929 10:59:28.944350 30881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance:
uuid: "d4bac528335c4b85b5d162c264194074"
format_stamp: "Formatted at 2026-09-29 10:59:28 on dist-test-slave-2lzq"
server_key: "cc04052a83e5232b8863ab7787113662"
server_key_iv: "61dd33f7372b6e06e476360e8296000d"
server_key_version: "encryptionkey@0"
I20260929 10:59:28.945726 30881 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 10:59:28.946465 30897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:28.946630 30881 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:28.946734 30881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-1/wal
uuid: "d4bac528335c4b85b5d162c264194074"
format_stamp: "Formatted at 2026-09-29 10:59:28 on dist-test-slave-2lzq"
server_key: "cc04052a83e5232b8863ab7787113662"
server_key_iv: "61dd33f7372b6e06e476360e8296000d"
server_key_version: "encryptionkey@0"
I20260929 10:59:28.946808 30881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:28.965507 30881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:28.967855 30881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:28.968024 30881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:28.968268 30881 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:59:28.968588 30881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:28.968645 30881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:28.968684 30881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:28.968712 30881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:28.974552 30881 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.194:38723
I20260929 10:59:28.974606 31011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.194:38723 every 8 connection(s)
I20260929 10:59:28.974903 30881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
Sep 29 10:59:28 dist-test-slave-2lzq krb5kdc[30640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679568, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:28.983304 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 30881
I20260929 10:59:28.983421 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance
I20260929 10:59:28.983690 27107 external_mini_cluster.cc:1428] Setting key e62e2f00a9cf0901a249815dad3b1c48
I20260929 10:59:28.984184 30743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:28.975634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:59489 (local address 127.26.120.254:44111)
0929 10:59:28.975679 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:28.975681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:28.975906 (+ 225us) server_negotiation.cc:407] Connection header received
0929 10:59:28.976863 (+ 957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:28.976864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:28.976877 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:28.976931 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:28.977581 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:28.978163 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:28.979170 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:28.979370 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:28.979448 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:28.981449 (+ 2001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:28.981450 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:28.981450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:28.981458 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:28.982670 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:28.983037 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:28.983038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:28.983038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:28.983102 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:28.983403 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:28.983404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:28.983405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:28.983574 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:28.983635 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:28.983996 (+ 361us) server_negotiation.cc:299] Negotiation successful
0929 10:59:28.984090 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 10:59:28.984393 31014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:28.975603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44111 (local address 127.26.120.194:59489)
0929 10:59:28.975786 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:28.975820 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:28.976736 (+ 916us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:28.977000 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:28.977004 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:28.977267 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:28.977468 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:28.977472 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:28.978252 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:28.978253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:28.979068 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:28.979069 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:28.979452 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:28.979453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:28.979589 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:28.980038 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:28.980045 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:28.981362 (+ 1317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:28.982743 (+ 1381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:28.982749 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:28.982751 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:28.982965 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:28.983167 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:28.983168 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:28.983168 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:28.983317 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:28.983645 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:28.983653 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:28.983824 (+ 171us) client_negotiation.cc:769] Sending connection context
0929 10:59:28.983970 (+ 146us) client_negotiation.cc:240] Negotiation successful
0929 10:59:28.984149 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":96,"threads_started":1}
I20260929 10:59:28.984838 31012 heartbeater.cc:347] Connected to a master server at 127.26.120.254:44111
I20260929 10:59:28.984943 31012 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:28.985144 31012 heartbeater.cc:511] Master 127.26.120.254:44111 requested a full tablet report, sending...
I20260929 10:59:28.985493 30685 ts_manager.cc:194] Registered new tserver with Master: d4bac528335c4b85b5d162c264194074 (127.26.120.194:38723)
I20260929 10:59:28.986013 30685 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.194@KRBTEST.COM'} at 127.26.120.194:59489
WARNING: no policy specified for kudu/127.26.120.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab.
Entry for principal kudu/127.26.120.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab.
I20260929 10:59:29.007153 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.195:0
--local_ip_for_outbound_sockets=127.26.120.195
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:44111
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:43079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:29.088371 31017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:29.088644 31017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:29.088689 31017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:29.090005 31017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:29.090106 31017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:29.090163 31017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.195
I20260929 10:59:29.091890 31017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:43079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:44111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.195
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:29.092180 31017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:29.092433 31017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:29.097815 31025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:29.097833 31017 server_base.cc:1034] running on GCE node
W20260929 10:59:29.097841 31022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:29.097816 31023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:29.098237 31017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:29.098524 31017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:29.099695 31017 hybrid_clock.cc:648] HybridClock initialized: now 1790679569099676 us; error 32 us; skew 500 ppm
Sep 29 10:59:29 dist-test-slave-2lzq krb5kdc[30640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679569, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:29.102387 31017 init.cc:377] Logged in from keytab as kudu/127.26.120.195@KRBTEST.COM (short username kudu)
I20260929 10:59:29.102772 31017 webserver.cc:460] Webserver started at http://127.26.120.195:37177/ using document root <none> and password file <none>
I20260929 10:59:29.102967 31017 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:29.103005 31017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:29.103107 31017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:29.104585 31017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-2/data/instance:
uuid: "f184510555834fd6a5791a7e508a275b"
format_stamp: "Formatted at 2026-09-29 10:59:29 on dist-test-slave-2lzq"
server_key: "9e665335f0124b130c81a0e681a0705a"
server_key_iv: "f1f1a582f37d06230c832bb4f1e4e91d"
server_key_version: "encryptionkey@0"
I20260929 10:59:29.104890 31017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance:
uuid: "f184510555834fd6a5791a7e508a275b"
format_stamp: "Formatted at 2026-09-29 10:59:29 on dist-test-slave-2lzq"
server_key: "9e665335f0124b130c81a0e681a0705a"
server_key_iv: "f1f1a582f37d06230c832bb4f1e4e91d"
server_key_version: "encryptionkey@0"
I20260929 10:59:29.106210 31017 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:59:29.106913 31032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:29.107092 31017 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:59:29.107187 31017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-2/wal
uuid: "f184510555834fd6a5791a7e508a275b"
format_stamp: "Formatted at 2026-09-29 10:59:29 on dist-test-slave-2lzq"
server_key: "9e665335f0124b130c81a0e681a0705a"
server_key_iv: "f1f1a582f37d06230c832bb4f1e4e91d"
server_key_version: "encryptionkey@0"
I20260929 10:59:29.107276 31017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:29.119699 31017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:29.122246 31017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:29.122407 31017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:29.122653 31017 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:59:29.122970 31017 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:29.123049 31017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:29.123093 31017 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:29.123121 31017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:29.128795 31017 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.195:41321
I20260929 10:59:29.128870 31146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.195:41321 every 8 connection(s)
I20260929 10:59:29.129204 31017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
I20260929 10:59:29.131676 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 31017
I20260929 10:59:29.131786 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance
I20260929 10:59:29.131960 27107 external_mini_cluster.cc:1428] Setting key b44c791fda38613926ab8accab8a5a70
Sep 29 10:59:29 dist-test-slave-2lzq krb5kdc[30640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679569, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:29.138398 30743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:29.129961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:53851 (local address 127.26.120.254:44111)
0929 10:59:29.130032 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:29.130034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:29.130276 (+ 242us) server_negotiation.cc:407] Connection header received
0929 10:59:29.131183 (+ 907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:29.131184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:29.131195 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:29.131254 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:29.132144 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:29.132764 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:29.133730 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:29.133911 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:29.133961 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:29.136011 (+ 2050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:29.136012 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:29.136012 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:29.136019 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:29.137204 (+ 1185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:29.137566 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:29.137567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:29.137568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:29.137617 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:29.137825 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:29.137826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:29.137826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:29.137923 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:29.137975 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:29.138263 (+ 288us) server_negotiation.cc:299] Negotiation successful
0929 10:59:29.138328 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 10:59:29.138545 31149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:29.129930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44111 (local address 127.26.120.195:53851)
0929 10:59:29.130147 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:29.130190 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:29.131089 (+ 899us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:29.131312 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:29.131316 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:29.131674 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:29.131969 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:29.131976 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:29.132833 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:29.132834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:29.133658 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:29.133660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:29.133979 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:29.133980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:29.134138 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:29.134590 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:29.134598 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:29.135927 (+ 1329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:29.137273 (+ 1346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:29.137278 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:29.137280 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:29.137497 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:29.137676 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:29.137676 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:29.137677 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:29.137770 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:29.137979 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:29.137983 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:29.138149 (+ 166us) client_negotiation.cc:769] Sending connection context
0929 10:59:29.138233 (+ 84us) client_negotiation.cc:240] Negotiation successful
0929 10:59:29.138347 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":103,"threads_started":1}
I20260929 10:59:29.139019 31147 heartbeater.cc:347] Connected to a master server at 127.26.120.254:44111
I20260929 10:59:29.139216 31147 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:29.139403 31147 heartbeater.cc:511] Master 127.26.120.254:44111 requested a full tablet report, sending...
I20260929 10:59:29.139818 30685 ts_manager.cc:194] Registered new tserver with Master: f184510555834fd6a5791a7e508a275b (127.26.120.195:41321)
I20260929 10:59:29.140218 30685 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.195@KRBTEST.COM'} at 127.26.120.195:53851
I20260929 10:59:29.143623 27107 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 10:59:29 dist-test-slave-2lzq krb5kdc[30640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 10:59:29 dist-test-slave-2lzq krb5kdc[30640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.193@KRBTEST.COM
I20260929 10:59:29.167090 31154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:29.159177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56206 (local address 127.26.120.193:32833)
0929 10:59:29.159359 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:29.159366 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:29.159376 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:59:29.159438 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:29.159441 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:29.159461 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:29.159534 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:29.160210 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:29.161101 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:29.161951 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:29.162373 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:29.162444 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:29.164535 (+ 2091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:29.164542 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:29.164543 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:29.164549 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:29.165928 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:29.166329 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:29.166332 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:29.166332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:29.166399 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:29.166564 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:29.166565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:29.166565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:29.166688 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:29.166791 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:29.166891 (+ 100us) server_negotiation.cc:299] Negotiation successful
0929 10:59:29.166951 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":94,"threads_started":1}
Sep 29 10:59:29 dist-test-slave-2lzq krb5kdc[30640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:29.174541 30743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:29.168431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54650 (local address 127.26.120.254:44111)
0929 10:59:29.168490 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:29.168492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:29.168500 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:59:29.168542 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:29.168543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:29.168552 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:29.168596 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:29.169011 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:29.169484 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:29.170161 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:29.170341 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:29.170390 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:29.172372 (+ 1982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:29.172374 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:29.172374 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:29.172381 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:29.173506 (+ 1125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:29.173787 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:29.173787 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:29.173788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:29.173833 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:29.173993 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:29.173993 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:29.173993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:29.174184 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:29.174261 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:29.174354 (+ 93us) server_negotiation.cc:299] Negotiation successful
0929 10:59:29.174414 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260929 10:59:29.174803 30685 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54650
I20260929 10:59:29.175175 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 30747
I20260929 10:59:29.179821 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 30881
I20260929 10:59:29.184522 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 31017
I20260929 10:59:29.188776 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 30653
2026-09-29T10:59:29Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1812 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:59:29 dist-test-slave-2lzq krb5kdc[31159](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:59:29 dist-test-slave-2lzq krb5kdc[31159](info): set up 2 sockets
Sep 29 10:59:29 dist-test-slave-2lzq krb5kdc[31159](info): commencing operation
krb5kdc: starting...
W20260929 10:59:30.226256 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 10:59:30 dist-test-slave-2lzq krb5kdc[31159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:59:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:59:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.120.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
I20260929 10:59:30.345939 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:44411
--webserver_interface=127.26.120.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:40509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:44411
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:30.425268 31172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:30.425526 31172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:30.425565 31172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:30.426721 31172 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:59:30.426793 31172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:30.426818 31172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:30.426847 31172 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:59:30.426872 31172 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:59:30.428341 31172 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:40509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:44411
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:44411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:30.428627 31172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:30.428845 31172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:30.433774 31178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:30.433796 31180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:30.433867 31172 server_base.cc:1034] running on GCE node
W20260929 10:59:30.433838 31177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:30.434230 31172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:30.434535 31172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:30.435700 31172 hybrid_clock.cc:648] HybridClock initialized: now 1790679570435696 us; error 38 us; skew 500 ppm
Sep 29 10:59:30 dist-test-slave-2lzq krb5kdc[31159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679570, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:30.438045 31172 init.cc:377] Logged in from keytab as kudu/127.26.120.254@KRBTEST.COM (short username kudu)
I20260929 10:59:30.438455 31172 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:30.438532 31172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:30.438668 31172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:30.439708 31172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "42b1e8f08f7a4d22a4f13b706add932e"
format_stamp: "Formatted at 2026-09-29 10:59:30 on dist-test-slave-2lzq"
server_key: "b48bbdd6fc27c9ee2464a3ecac2f9e18"
server_key_iv: "d3a3c56a70ea2b5c9f23561270189ba7"
server_key_version: "encryptionkey@0"
I20260929 10:59:30.440073 31172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "42b1e8f08f7a4d22a4f13b706add932e"
format_stamp: "Formatted at 2026-09-29 10:59:30 on dist-test-slave-2lzq"
server_key: "b48bbdd6fc27c9ee2464a3ecac2f9e18"
server_key_iv: "d3a3c56a70ea2b5c9f23561270189ba7"
server_key_version: "encryptionkey@0"
I20260929 10:59:30.441377 31172 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 10:59:30.442171 31187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:30.442364 31172 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:59:30.442461 31172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "42b1e8f08f7a4d22a4f13b706add932e"
format_stamp: "Formatted at 2026-09-29 10:59:30 on dist-test-slave-2lzq"
server_key: "b48bbdd6fc27c9ee2464a3ecac2f9e18"
server_key_iv: "d3a3c56a70ea2b5c9f23561270189ba7"
server_key_version: "encryptionkey@0"
I20260929 10:59:30.442551 31172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:30.459053 31172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:30.460961 31172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:30.461127 31172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:30.465090 31172 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:44411
I20260929 10:59:30.465121 31239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:44411 every 8 connection(s)
I20260929 10:59:30.465407 31172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 10:59:30.466325 31240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:30.468520 31240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42b1e8f08f7a4d22a4f13b706add932e: Bootstrap starting.
I20260929 10:59:30.469049 31240 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42b1e8f08f7a4d22a4f13b706add932e: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:30.469285 31240 log.cc:824] T 00000000000000000000000000000000 P 42b1e8f08f7a4d22a4f13b706add932e: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:30.469887 31240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42b1e8f08f7a4d22a4f13b706add932e: No bootstrap required, opened a new log
I20260929 10:59:30.470602 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 31172
I20260929 10:59:30.470712 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 10:59:30.470897 27107 external_mini_cluster.cc:1428] Setting key 9ea197fcd60de3c40e4e89c68605b432
I20260929 10:59:30.471839 31240 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42b1e8f08f7a4d22a4f13b706add932e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42b1e8f08f7a4d22a4f13b706add932e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44411 } }
I20260929 10:59:30.472013 31240 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42b1e8f08f7a4d22a4f13b706add932e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:30.472064 31240 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42b1e8f08f7a4d22a4f13b706add932e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42b1e8f08f7a4d22a4f13b706add932e, State: Initialized, Role: FOLLOWER
I20260929 10:59:30.472222 31240 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42b1e8f08f7a4d22a4f13b706add932e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42b1e8f08f7a4d22a4f13b706add932e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44411 } }
I20260929 10:59:30.472319 31240 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42b1e8f08f7a4d22a4f13b706add932e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:59:30.472349 31240 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42b1e8f08f7a4d22a4f13b706add932e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:59:30.472410 31240 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42b1e8f08f7a4d22a4f13b706add932e [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:30.473074 31240 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42b1e8f08f7a4d22a4f13b706add932e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42b1e8f08f7a4d22a4f13b706add932e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44411 } }
I20260929 10:59:30.473179 31240 leader_election.cc:302] T 00000000000000000000000000000000 P 42b1e8f08f7a4d22a4f13b706add932e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42b1e8f08f7a4d22a4f13b706add932e; no voters:
I20260929 10:59:30.473331 31240 leader_election.cc:288] T 00000000000000000000000000000000 P 42b1e8f08f7a4d22a4f13b706add932e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:59:30.473407 31245 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42b1e8f08f7a4d22a4f13b706add932e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:59:30.473544 31245 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42b1e8f08f7a4d22a4f13b706add932e [term 1 LEADER]: Becoming Leader. State: Replica: 42b1e8f08f7a4d22a4f13b706add932e, State: Running, Role: LEADER
I20260929 10:59:30.473572 31240 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42b1e8f08f7a4d22a4f13b706add932e [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:59:30.473649 31245 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42b1e8f08f7a4d22a4f13b706add932e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42b1e8f08f7a4d22a4f13b706add932e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44411 } }
I20260929 10:59:30.473923 31245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42b1e8f08f7a4d22a4f13b706add932e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42b1e8f08f7a4d22a4f13b706add932e. Latest consensus state: current_term: 1 leader_uuid: "42b1e8f08f7a4d22a4f13b706add932e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42b1e8f08f7a4d22a4f13b706add932e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44411 } } }
I20260929 10:59:30.474014 31245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42b1e8f08f7a4d22a4f13b706add932e [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:30.473941 31247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42b1e8f08f7a4d22a4f13b706add932e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42b1e8f08f7a4d22a4f13b706add932e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42b1e8f08f7a4d22a4f13b706add932e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44411 } } }
I20260929 10:59:30.474157 31247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42b1e8f08f7a4d22a4f13b706add932e [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:30.474236 31252 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:59:30.474710 31252 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:59:30.476236 31252 catalog_manager.cc:1347] Generated new cluster ID: 4bd32e4b586f4ae19dc5f2597741d8a1
I20260929 10:59:30.476286 31252 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 10:59:30 dist-test-slave-2lzq krb5kdc[31159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:30.483343 31244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:30.471409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57080 (local address 127.26.120.254:44411)
0929 10:59:30.471636 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:30.471647 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:30.471682 (+ 35us) server_negotiation.cc:407] Connection header received
0929 10:59:30.471798 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:30.471804 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:30.471860 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:30.472024 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:30.472785 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:30.474294 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:30.475342 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:30.475958 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:30.476064 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:30.478939 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:30.478944 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:30.478944 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:30.478956 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:30.481188 (+ 2232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:30.481599 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:30.481603 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:30.481603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:30.481725 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:30.481944 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:30.481945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:30.481946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:30.482308 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:30.482499 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:30.482808 (+ 309us) server_negotiation.cc:299] Negotiation successful
0929 10:59:30.483118 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":98,"threads_started":1}
I20260929 10:59:30.493796 31252 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:59:30.494566 31252 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:59:30.499015 31252 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42b1e8f08f7a4d22a4f13b706add932e: Generated new TSK 0
I20260929 10:59:30.499202 31252 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.120.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab.
Entry for principal kudu/127.26.120.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab.
I20260929 10:59:30.525238 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.193:0
--local_ip_for_outbound_sockets=127.26.120.193
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:44411
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:40509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:30.604969 31266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:30.605199 31266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:30.605242 31266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:30.606441 31266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:30.606508 31266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:30.606559 31266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.193
I20260929 10:59:30.608052 31266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:40509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:44411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.193
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:30.608316 31266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:30.608569 31266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:30.613505 31272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:30.613638 31266 server_base.cc:1034] running on GCE node
W20260929 10:59:30.613485 31274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:30.613485 31271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:30.613924 31266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:30.614241 31266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:30.615448 31266 hybrid_clock.cc:648] HybridClock initialized: now 1790679570615433 us; error 29 us; skew 500 ppm
Sep 29 10:59:30 dist-test-slave-2lzq krb5kdc[31159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679570, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:30.618359 31266 init.cc:377] Logged in from keytab as kudu/127.26.120.193@KRBTEST.COM (short username kudu)
I20260929 10:59:30.618779 31266 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:30.618834 31266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:30.618933 31266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:30.620062 31266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-0/data/instance:
uuid: "0a7bde56ef664c0fbb2dc379102e8f08"
format_stamp: "Formatted at 2026-09-29 10:59:30 on dist-test-slave-2lzq"
server_key: "42d32309e8c32fcdd747c911a1be8f73"
server_key_iv: "f4abb882a8f88c8e115b1c82f1d139b6"
server_key_version: "encryptionkey@0"
I20260929 10:59:30.620397 31266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance:
uuid: "0a7bde56ef664c0fbb2dc379102e8f08"
format_stamp: "Formatted at 2026-09-29 10:59:30 on dist-test-slave-2lzq"
server_key: "42d32309e8c32fcdd747c911a1be8f73"
server_key_iv: "f4abb882a8f88c8e115b1c82f1d139b6"
server_key_version: "encryptionkey@0"
I20260929 10:59:30.621714 31266 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 10:59:30.622607 31280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:30.622821 31266 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 10:59:30.622910 31266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-0/wal
uuid: "0a7bde56ef664c0fbb2dc379102e8f08"
format_stamp: "Formatted at 2026-09-29 10:59:30 on dist-test-slave-2lzq"
server_key: "42d32309e8c32fcdd747c911a1be8f73"
server_key_iv: "f4abb882a8f88c8e115b1c82f1d139b6"
server_key_version: "encryptionkey@0"
I20260929 10:59:30.622967 31266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:30.644575 31266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:30.646610 31266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:30.646771 31266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:30.647007 31266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:59:30.647330 31266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:30.647363 31266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:30.647385 31266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:30.647434 31266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:30.653010 31266 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.193:46581
I20260929 10:59:30.653082 31394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.193:46581 every 8 connection(s)
I20260929 10:59:30.653365 31266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
Sep 29 10:59:30 dist-test-slave-2lzq krb5kdc[31159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679570, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:30.660274 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 31266
I20260929 10:59:30.660388 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance
I20260929 10:59:30.660578 27107 external_mini_cluster.cc:1428] Setting key 68f90923c2e905e7fd6de33b8b94a559
I20260929 10:59:30.662281 31244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:30.654136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:56495 (local address 127.26.120.254:44411)
0929 10:59:30.654184 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:30.654186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:30.654373 (+ 187us) server_negotiation.cc:407] Connection header received
0929 10:59:30.654845 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:30.654846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:30.654860 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:30.654910 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:30.655805 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:30.656297 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:30.657241 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:30.657450 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:30.657500 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:30.659451 (+ 1951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:30.659453 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:30.659453 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:30.659459 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:30.660860 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:30.661311 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:30.661312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:30.661312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:30.661362 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:30.661635 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:30.661636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:30.661636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:30.661733 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:30.661782 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:30.662154 (+ 372us) server_negotiation.cc:299] Negotiation successful
0929 10:59:30.662222 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 10:59:30.662576 31397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:30.654126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44411 (local address 127.26.120.193:56495)
0929 10:59:30.654266 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:30.654302 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:30.654756 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:30.654971 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:30.654973 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:30.655238 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:30.655727 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:30.655732 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:30.656373 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:30.656374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:30.657167 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:30.657168 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:30.657521 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:30.657521 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:30.657620 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:30.658087 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:30.658094 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:30.659369 (+ 1275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:30.660959 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:30.660966 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:30.660968 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:30.661236 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:30.661428 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:30.661429 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:30.661430 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:30.661570 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:30.661797 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:30.661803 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:30.661978 (+ 175us) client_negotiation.cc:769] Sending connection context
0929 10:59:30.662124 (+ 146us) client_negotiation.cc:240] Negotiation successful
0929 10:59:30.662304 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":75,"threads_started":1}
I20260929 10:59:30.663129 31395 heartbeater.cc:347] Connected to a master server at 127.26.120.254:44411
I20260929 10:59:30.663236 31395 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:30.663393 31395 heartbeater.cc:511] Master 127.26.120.254:44411 requested a full tablet report, sending...
I20260929 10:59:30.663893 31204 ts_manager.cc:194] Registered new tserver with Master: 0a7bde56ef664c0fbb2dc379102e8f08 (127.26.120.193:46581)
I20260929 10:59:30.664491 31204 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.193@KRBTEST.COM'} at 127.26.120.193:56495
WARNING: no policy specified for kudu/127.26.120.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab.
Entry for principal kudu/127.26.120.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab.
I20260929 10:59:30.684765 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.194:0
--local_ip_for_outbound_sockets=127.26.120.194
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:44411
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:40509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:30.767292 31400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:30.767483 31400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:30.767506 31400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:30.768587 31400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:30.768621 31400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:30.768646 31400 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.194
I20260929 10:59:30.770033 31400 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:40509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:44411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.194
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:30.770224 31400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:30.770391 31400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:30.775722 31408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:30.775846 31406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:30.775848 31405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:30.776041 31400 server_base.cc:1034] running on GCE node
I20260929 10:59:30.776249 31400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:30.776484 31400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:30.777654 31400 hybrid_clock.cc:648] HybridClock initialized: now 1790679570777652 us; error 38 us; skew 500 ppm
Sep 29 10:59:30 dist-test-slave-2lzq krb5kdc[31159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679570, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:30.780011 31400 init.cc:377] Logged in from keytab as kudu/127.26.120.194@KRBTEST.COM (short username kudu)
I20260929 10:59:30.780380 31400 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:30.780452 31400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:30.780586 31400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:30.781616 31400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-1/data/instance:
uuid: "c7b8cbdb0dbb4244b897e4324123b13f"
format_stamp: "Formatted at 2026-09-29 10:59:30 on dist-test-slave-2lzq"
server_key: "03240faf0e2b4f2f98c2ee4cf3f06b72"
server_key_iv: "f6e9bbdaf323b2a93d7bab4f4cc44176"
server_key_version: "encryptionkey@0"
I20260929 10:59:30.781980 31400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance:
uuid: "c7b8cbdb0dbb4244b897e4324123b13f"
format_stamp: "Formatted at 2026-09-29 10:59:30 on dist-test-slave-2lzq"
server_key: "03240faf0e2b4f2f98c2ee4cf3f06b72"
server_key_iv: "f6e9bbdaf323b2a93d7bab4f4cc44176"
server_key_version: "encryptionkey@0"
I20260929 10:59:30.783237 31400 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:59:30.783912 31414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:30.784093 31400 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:59:30.784186 31400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-1/wal
uuid: "c7b8cbdb0dbb4244b897e4324123b13f"
format_stamp: "Formatted at 2026-09-29 10:59:30 on dist-test-slave-2lzq"
server_key: "03240faf0e2b4f2f98c2ee4cf3f06b72"
server_key_iv: "f6e9bbdaf323b2a93d7bab4f4cc44176"
server_key_version: "encryptionkey@0"
I20260929 10:59:30.784269 31400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:30.797138 31400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:30.799050 31400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:30.799214 31400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:30.799453 31400 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:59:30.799786 31400 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:30.799839 31400 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:30.799873 31400 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:30.799901 31400 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:30.805657 31400 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.194:42997
I20260929 10:59:30.805696 31527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.194:42997 every 8 connection(s)
I20260929 10:59:30.805971 31400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
I20260929 10:59:30.809187 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 31400
I20260929 10:59:30.809312 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance
I20260929 10:59:30.809499 27107 external_mini_cluster.cc:1428] Setting key 290e258524016505b2e8c466d9da4158
Sep 29 10:59:30 dist-test-slave-2lzq krb5kdc[31159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679570, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:30.815642 31530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:30.806714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44411 (local address 127.26.120.194:47535)
0929 10:59:30.806859 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:30.806899 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:30.807358 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:30.807600 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:30.807603 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:30.807845 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:30.808335 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:30.808339 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:30.808985 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:30.808986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:30.810193 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:30.810196 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:30.810681 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:30.810681 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:30.810781 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:30.811299 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:30.811307 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:30.812660 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:30.814284 (+ 1624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:30.814291 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:30.814293 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:30.814556 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:30.814795 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:30.814796 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:30.814796 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:30.814892 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:30.815183 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:30.815187 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:30.815294 (+ 107us) client_negotiation.cc:769] Sending connection context
0929 10:59:30.815373 (+ 79us) client_negotiation.cc:240] Negotiation successful
0929 10:59:30.815484 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":77,"threads_started":1}
I20260929 10:59:30.815657 31244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:30.806745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:47535 (local address 127.26.120.254:44411)
0929 10:59:30.806815 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:30.806817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:30.807012 (+ 195us) server_negotiation.cc:407] Connection header received
0929 10:59:30.807471 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:30.807472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:30.807482 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:30.807536 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:30.808413 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:30.808907 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:30.810294 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:30.810597 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:30.810673 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:30.812775 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:30.812777 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:30.812777 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:30.812784 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:30.814196 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:30.814633 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:30.814635 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:30.814635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:30.814722 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:30.814959 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:30.814960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:30.814961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:30.815115 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:30.815192 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:30.815424 (+ 232us) server_negotiation.cc:299] Negotiation successful
0929 10:59:30.815521 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 10:59:30.816169 31528 heartbeater.cc:347] Connected to a master server at 127.26.120.254:44411
I20260929 10:59:30.816269 31528 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:30.816440 31528 heartbeater.cc:511] Master 127.26.120.254:44411 requested a full tablet report, sending...
I20260929 10:59:30.816797 31204 ts_manager.cc:194] Registered new tserver with Master: c7b8cbdb0dbb4244b897e4324123b13f (127.26.120.194:42997)
I20260929 10:59:30.817179 31204 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.194@KRBTEST.COM'} at 127.26.120.194:47535
WARNING: no policy specified for kudu/127.26.120.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab.
Entry for principal kudu/127.26.120.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab.
I20260929 10:59:30.836814 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.195:0
--local_ip_for_outbound_sockets=127.26.120.195
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:44411
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:40509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:30.917204 31533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:30.917439 31533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:30.917464 31533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:30.918694 31533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:30.918741 31533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:30.918792 31533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.195
I20260929 10:59:30.920325 31533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:40509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:44411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.195
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:30.920583 31533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:30.920814 31533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:30.925673 31538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:30.925743 31533 server_base.cc:1034] running on GCE node
W20260929 10:59:30.925678 31539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:30.925725 31541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:30.926146 31533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:30.926349 31533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:30.927520 31533 hybrid_clock.cc:648] HybridClock initialized: now 1790679570927506 us; error 58 us; skew 500 ppm
Sep 29 10:59:30 dist-test-slave-2lzq krb5kdc[31159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679570, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:30.929921 31533 init.cc:377] Logged in from keytab as kudu/127.26.120.195@KRBTEST.COM (short username kudu)
I20260929 10:59:30.930335 31533 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:30.930408 31533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:30.930553 31533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:30.931670 31533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-2/data/instance:
uuid: "bd9f3615c1574fffbc439f5f55f6de7e"
format_stamp: "Formatted at 2026-09-29 10:59:30 on dist-test-slave-2lzq"
server_key: "28a0fce2d226b4ddd61fe91de83596c5"
server_key_iv: "b378dd2ff93928ca8eda40247e306e45"
server_key_version: "encryptionkey@0"
I20260929 10:59:30.932041 31533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance:
uuid: "bd9f3615c1574fffbc439f5f55f6de7e"
format_stamp: "Formatted at 2026-09-29 10:59:30 on dist-test-slave-2lzq"
server_key: "28a0fce2d226b4ddd61fe91de83596c5"
server_key_iv: "b378dd2ff93928ca8eda40247e306e45"
server_key_version: "encryptionkey@0"
I20260929 10:59:30.933324 31533 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 10:59:30.934124 31547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:30.934319 31533 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:59:30.934412 31533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-2/wal
uuid: "bd9f3615c1574fffbc439f5f55f6de7e"
format_stamp: "Formatted at 2026-09-29 10:59:30 on dist-test-slave-2lzq"
server_key: "28a0fce2d226b4ddd61fe91de83596c5"
server_key_iv: "b378dd2ff93928ca8eda40247e306e45"
server_key_version: "encryptionkey@0"
I20260929 10:59:30.934494 31533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:30.950860 31533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:30.952682 31533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:30.952847 31533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:30.953086 31533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:59:30.953413 31533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:30.953474 31533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:30.953539 31533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:30.953572 31533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:30.959354 31533 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.195:44105
I20260929 10:59:30.959389 31660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.195:44105 every 8 connection(s)
I20260929 10:59:30.959657 31533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
I20260929 10:59:30.961508 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 31533
I20260929 10:59:30.961623 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance
I20260929 10:59:30.961792 27107 external_mini_cluster.cc:1428] Setting key 028ad6c8f80c9ef7fc35c337c21fbcef
Sep 29 10:59:30 dist-test-slave-2lzq krb5kdc[31159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679570, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:30.968926 31244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:30.960368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:43803 (local address 127.26.120.254:44411)
0929 10:59:30.960454 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:30.960456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:30.960650 (+ 194us) server_negotiation.cc:407] Connection header received
0929 10:59:30.961104 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:30.961105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:30.961115 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:30.961165 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:30.962350 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:30.963009 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:30.963976 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:30.964154 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:30.964204 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:30.966469 (+ 2265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:30.966470 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:30.966470 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:30.966476 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:30.967823 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:30.968155 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:30.968155 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:30.968156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:30.968202 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:30.968402 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:30.968402 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:30.968403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:30.968495 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:30.968552 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:30.968789 (+ 237us) server_negotiation.cc:299] Negotiation successful
0929 10:59:30.968864 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 10:59:30.969146 31663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:30.960321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44411 (local address 127.26.120.195:43803)
0929 10:59:30.960526 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:30.960565 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:30.961024 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:30.961219 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:30.961223 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:30.961485 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:30.962185 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:30.962192 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:30.963084 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:30.963085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:30.963907 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:30.963909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:30.964355 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:30.964356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:30.964469 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:30.964958 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:30.964966 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:30.966388 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:30.967891 (+ 1503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:30.967895 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:30.967897 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:30.968095 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:30.968256 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:30.968257 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:30.968257 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:30.968348 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:30.968564 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:30.968568 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:30.968685 (+ 117us) client_negotiation.cc:769] Sending connection context
0929 10:59:30.968764 (+ 79us) client_negotiation.cc:240] Negotiation successful
0929 10:59:30.968896 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":98,"threads_started":1}
I20260929 10:59:30.969632 31661 heartbeater.cc:347] Connected to a master server at 127.26.120.254:44411
I20260929 10:59:30.969735 31661 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:30.969877 31661 heartbeater.cc:511] Master 127.26.120.254:44411 requested a full tablet report, sending...
I20260929 10:59:30.970299 31204 ts_manager.cc:194] Registered new tserver with Master: bd9f3615c1574fffbc439f5f55f6de7e (127.26.120.195:44105)
I20260929 10:59:30.970659 31204 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.195@KRBTEST.COM'} at 127.26.120.195:43803
I20260929 10:59:30.973538 27107 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:59:30.980484 31244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:30.974916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57082 (local address 127.26.120.254:44411)
0929 10:59:30.974978 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:30.974980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:30.975237 (+ 257us) server_negotiation.cc:407] Connection header received
0929 10:59:30.975274 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:30.975274 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:30.975283 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:30.975325 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:30.975828 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:30.976284 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:30.976936 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:30.977116 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:30.977165 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:30.978275 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:30.978276 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:30.978277 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:30.978283 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:30.979616 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:30.979877 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:30.979878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:30.979878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:30.979920 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:30.980061 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:30.980061 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:30.980061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:30.980178 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:30.980246 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:30.980327 (+ 81us) server_negotiation.cc:299] Negotiation successful
0929 10:59:30.980381 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260929 10:59:30.981946 31204 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57082:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:59:30.983560 31204 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:59:30.989924 31673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:30.985839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36438 (local address 127.26.120.193:46581)
0929 10:59:30.986236 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:30.986240 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:30.986249 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:59:30.986283 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:30.986285 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:30.986299 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:30.986540 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:30.986625 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:30.987807 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:30.988981 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:30.989506 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:30.989618 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:30.989680 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:30.989810 (+ 130us) server_negotiation.cc:299] Negotiation successful
0929 10:59:30.989848 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":71,"threads_started":1}
I20260929 10:59:30.989910 31669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:30.985644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:46581 (local address 127.0.0.1:36438)
0929 10:59:30.985986 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:30.985995 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:30.986049 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:30.986393 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:30.986394 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:30.986396 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:30.986454 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:30.986455 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:30.987921 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:30.987922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:30.988897 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:30.988899 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:30.989619 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:30.989619 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:30.989699 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:30.989712 (+ 13us) client_negotiation.cc:769] Sending connection context
0929 10:59:30.989778 (+ 66us) client_negotiation.cc:240] Negotiation successful
0929 10:59:30.989834 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":1,"thread_start_us":68,"threads_started":1}
I20260929 10:59:30.990123 31674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:30.986016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47900 (local address 127.26.120.195:44105)
0929 10:59:30.986222 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:30.986226 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:30.986235 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:59:30.986296 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:30.986298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:30.986313 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:30.986391 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:30.986591 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:30.987379 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:30.988860 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:30.989596 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:30.989667 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:30.989716 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:30.989956 (+ 240us) server_negotiation.cc:299] Negotiation successful
0929 10:59:30.990000 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":111,"threads_started":1}
I20260929 10:59:30.990276 31672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:30.985868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:44105 (local address 127.0.0.1:47900)
0929 10:59:30.986029 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:30.986049 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:30.986125 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:30.986472 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:30.986473 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:30.986475 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:30.986505 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:30.986506 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:30.987482 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:30.987482 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:30.988769 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:30.988772 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:30.989752 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:30.989753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:30.989820 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:30.989824 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:59:30.989870 (+ 46us) client_negotiation.cc:240] Negotiation successful
0929 10:59:30.989913 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":39,"threads_started":1}
I20260929 10:59:30.990497 31670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:30.985642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:42997 (local address 127.0.0.1:60352)
0929 10:59:30.985918 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:30.985958 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:30.986049 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:30.986308 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:30.986311 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:30.986313 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:30.986387 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:30.986389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:30.987900 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:30.987901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:30.989000 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:30.989002 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:30.989899 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:30.989900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:30.990323 (+ 423us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:30.990330 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 10:59:30.990391 (+ 61us) client_negotiation.cc:240] Negotiation successful
0929 10:59:30.990437 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":37,"thread_start_us":76,"threads_started":1}
I20260929 10:59:30.990688 31671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:30.985741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60352 (local address 127.26.120.194:42997)
0929 10:59:30.985925 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:30.985931 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:30.986145 (+ 214us) server_negotiation.cc:407] Connection header received
0929 10:59:30.986191 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:30.986194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:30.986211 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:30.986288 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:30.986669 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:30.987790 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:30.989086 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:30.989762 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:30.989823 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:30.989869 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:30.990448 (+ 579us) server_negotiation.cc:299] Negotiation successful
0929 10:59:30.990602 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":85,"threads_started":1}
I20260929 10:59:30.990696 31328 tablet_service.cc:1467] Processing CreateTablet for tablet 8eefb8b3ab124b08913772a31ac88f07 (DEFAULT_TABLE table=test-table [id=f8518d625ce544c3a083796bc360830d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:30.990862 31595 tablet_service.cc:1467] Processing CreateTablet for tablet 8eefb8b3ab124b08913772a31ac88f07 (DEFAULT_TABLE table=test-table [id=f8518d625ce544c3a083796bc360830d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:30.991082 31328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8eefb8b3ab124b08913772a31ac88f07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:30.991138 31595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8eefb8b3ab124b08913772a31ac88f07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:30.991367 31462 tablet_service.cc:1467] Processing CreateTablet for tablet 8eefb8b3ab124b08913772a31ac88f07 (DEFAULT_TABLE table=test-table [id=f8518d625ce544c3a083796bc360830d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:30.991686 31462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8eefb8b3ab124b08913772a31ac88f07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:30.994321 31675 tablet_bootstrap.cc:492] T 8eefb8b3ab124b08913772a31ac88f07 P c7b8cbdb0dbb4244b897e4324123b13f: Bootstrap starting.
I20260929 10:59:30.994765 31677 tablet_bootstrap.cc:492] T 8eefb8b3ab124b08913772a31ac88f07 P bd9f3615c1574fffbc439f5f55f6de7e: Bootstrap starting.
I20260929 10:59:30.994984 31675 tablet_bootstrap.cc:654] T 8eefb8b3ab124b08913772a31ac88f07 P c7b8cbdb0dbb4244b897e4324123b13f: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:30.995340 31677 tablet_bootstrap.cc:654] T 8eefb8b3ab124b08913772a31ac88f07 P bd9f3615c1574fffbc439f5f55f6de7e: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:30.995522 31675 log.cc:824] T 8eefb8b3ab124b08913772a31ac88f07 P c7b8cbdb0dbb4244b897e4324123b13f: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:30.995875 31677 log.cc:824] T 8eefb8b3ab124b08913772a31ac88f07 P bd9f3615c1574fffbc439f5f55f6de7e: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:30.996182 31676 tablet_bootstrap.cc:492] T 8eefb8b3ab124b08913772a31ac88f07 P 0a7bde56ef664c0fbb2dc379102e8f08: Bootstrap starting.
I20260929 10:59:30.996268 31675 tablet_bootstrap.cc:492] T 8eefb8b3ab124b08913772a31ac88f07 P c7b8cbdb0dbb4244b897e4324123b13f: No bootstrap required, opened a new log
I20260929 10:59:30.996336 31675 ts_tablet_manager.cc:1397] T 8eefb8b3ab124b08913772a31ac88f07 P c7b8cbdb0dbb4244b897e4324123b13f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:59:30.996747 31676 tablet_bootstrap.cc:654] T 8eefb8b3ab124b08913772a31ac88f07 P 0a7bde56ef664c0fbb2dc379102e8f08: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:30.997349 31676 log.cc:824] T 8eefb8b3ab124b08913772a31ac88f07 P 0a7bde56ef664c0fbb2dc379102e8f08: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:30.997912 31677 tablet_bootstrap.cc:492] T 8eefb8b3ab124b08913772a31ac88f07 P bd9f3615c1574fffbc439f5f55f6de7e: No bootstrap required, opened a new log
I20260929 10:59:30.997991 31677 ts_tablet_manager.cc:1397] T 8eefb8b3ab124b08913772a31ac88f07 P bd9f3615c1574fffbc439f5f55f6de7e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260929 10:59:30.998214 31676 tablet_bootstrap.cc:492] T 8eefb8b3ab124b08913772a31ac88f07 P 0a7bde56ef664c0fbb2dc379102e8f08: No bootstrap required, opened a new log
I20260929 10:59:30.998281 31676 ts_tablet_manager.cc:1397] T 8eefb8b3ab124b08913772a31ac88f07 P 0a7bde56ef664c0fbb2dc379102e8f08: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:59:30.998230 31675 raft_consensus.cc:348] T 8eefb8b3ab124b08913772a31ac88f07 P c7b8cbdb0dbb4244b897e4324123b13f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7b8cbdb0dbb4244b897e4324123b13f" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 42997 } } peers { permanent_uuid: "0a7bde56ef664c0fbb2dc379102e8f08" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46581 } } peers { permanent_uuid: "bd9f3615c1574fffbc439f5f55f6de7e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44105 } }
I20260929 10:59:30.998409 31675 raft_consensus.cc:374] T 8eefb8b3ab124b08913772a31ac88f07 P c7b8cbdb0dbb4244b897e4324123b13f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:30.998456 31675 raft_consensus.cc:729] T 8eefb8b3ab124b08913772a31ac88f07 P c7b8cbdb0dbb4244b897e4324123b13f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7b8cbdb0dbb4244b897e4324123b13f, State: Initialized, Role: FOLLOWER
I20260929 10:59:30.998576 31675 consensus_queue.cc:260] T 8eefb8b3ab124b08913772a31ac88f07 P c7b8cbdb0dbb4244b897e4324123b13f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7b8cbdb0dbb4244b897e4324123b13f" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 42997 } } peers { permanent_uuid: "0a7bde56ef664c0fbb2dc379102e8f08" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46581 } } peers { permanent_uuid: "bd9f3615c1574fffbc439f5f55f6de7e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44105 } }
I20260929 10:59:30.998855 31675 ts_tablet_manager.cc:1428] T 8eefb8b3ab124b08913772a31ac88f07 P c7b8cbdb0dbb4244b897e4324123b13f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 10:59:30.998986 31528 heartbeater.cc:503] Master 127.26.120.254:44411 was elected leader, sending a full tablet report...
I20260929 10:59:30.999969 31677 raft_consensus.cc:348] T 8eefb8b3ab124b08913772a31ac88f07 P bd9f3615c1574fffbc439f5f55f6de7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7b8cbdb0dbb4244b897e4324123b13f" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 42997 } } peers { permanent_uuid: "0a7bde56ef664c0fbb2dc379102e8f08" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46581 } } peers { permanent_uuid: "bd9f3615c1574fffbc439f5f55f6de7e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44105 } }
I20260929 10:59:31.000138 31677 raft_consensus.cc:374] T 8eefb8b3ab124b08913772a31ac88f07 P bd9f3615c1574fffbc439f5f55f6de7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:31.000182 31677 raft_consensus.cc:729] T 8eefb8b3ab124b08913772a31ac88f07 P bd9f3615c1574fffbc439f5f55f6de7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd9f3615c1574fffbc439f5f55f6de7e, State: Initialized, Role: FOLLOWER
I20260929 10:59:31.000300 31677 consensus_queue.cc:260] T 8eefb8b3ab124b08913772a31ac88f07 P bd9f3615c1574fffbc439f5f55f6de7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7b8cbdb0dbb4244b897e4324123b13f" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 42997 } } peers { permanent_uuid: "0a7bde56ef664c0fbb2dc379102e8f08" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46581 } } peers { permanent_uuid: "bd9f3615c1574fffbc439f5f55f6de7e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44105 } }
I20260929 10:59:31.000250 31676 raft_consensus.cc:348] T 8eefb8b3ab124b08913772a31ac88f07 P 0a7bde56ef664c0fbb2dc379102e8f08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7b8cbdb0dbb4244b897e4324123b13f" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 42997 } } peers { permanent_uuid: "0a7bde56ef664c0fbb2dc379102e8f08" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46581 } } peers { permanent_uuid: "bd9f3615c1574fffbc439f5f55f6de7e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44105 } }
I20260929 10:59:31.000389 31676 raft_consensus.cc:374] T 8eefb8b3ab124b08913772a31ac88f07 P 0a7bde56ef664c0fbb2dc379102e8f08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:31.000438 31676 raft_consensus.cc:729] T 8eefb8b3ab124b08913772a31ac88f07 P 0a7bde56ef664c0fbb2dc379102e8f08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a7bde56ef664c0fbb2dc379102e8f08, State: Initialized, Role: FOLLOWER
I20260929 10:59:31.000572 31661 heartbeater.cc:503] Master 127.26.120.254:44411 was elected leader, sending a full tablet report...
I20260929 10:59:31.000557 31676 consensus_queue.cc:260] T 8eefb8b3ab124b08913772a31ac88f07 P 0a7bde56ef664c0fbb2dc379102e8f08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7b8cbdb0dbb4244b897e4324123b13f" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 42997 } } peers { permanent_uuid: "0a7bde56ef664c0fbb2dc379102e8f08" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46581 } } peers { permanent_uuid: "bd9f3615c1574fffbc439f5f55f6de7e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44105 } }
I20260929 10:59:31.000820 31676 ts_tablet_manager.cc:1428] T 8eefb8b3ab124b08913772a31ac88f07 P 0a7bde56ef664c0fbb2dc379102e8f08: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 10:59:31.000814 31677 ts_tablet_manager.cc:1428] T 8eefb8b3ab124b08913772a31ac88f07 P bd9f3615c1574fffbc439f5f55f6de7e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 10:59:31.000835 31395 heartbeater.cc:503] Master 127.26.120.254:44411 was elected leader, sending a full tablet report...
W20260929 10:59:31.056612 31529 tablet.cc:2369] T 8eefb8b3ab124b08913772a31ac88f07 P c7b8cbdb0dbb4244b897e4324123b13f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:59:31.111150 31681 raft_consensus.cc:482] T 8eefb8b3ab124b08913772a31ac88f07 P c7b8cbdb0dbb4244b897e4324123b13f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:59:31.111316 31681 raft_consensus.cc:504] T 8eefb8b3ab124b08913772a31ac88f07 P c7b8cbdb0dbb4244b897e4324123b13f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7b8cbdb0dbb4244b897e4324123b13f" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 42997 } } peers { permanent_uuid: "0a7bde56ef664c0fbb2dc379102e8f08" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46581 } } peers { permanent_uuid: "bd9f3615c1574fffbc439f5f55f6de7e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44105 } }
I20260929 10:59:31.111703 31681 leader_election.cc:288] T 8eefb8b3ab124b08913772a31ac88f07 P c7b8cbdb0dbb4244b897e4324123b13f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a7bde56ef664c0fbb2dc379102e8f08 (127.26.120.193:46581), bd9f3615c1574fffbc439f5f55f6de7e (127.26.120.195:44105)
I20260929 10:59:31.114669 31530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:31.111872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:46581 (local address 127.26.120.194:38157)
0929 10:59:31.111936 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:31.111949 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:31.111999 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:31.112182 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:31.112183 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:31.112185 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:31.112257 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:31.112259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:31.112992 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:31.112993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:31.113821 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:31.113823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:31.114394 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:31.114395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:31.114474 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:31.114478 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:59:31.114547 (+ 69us) client_negotiation.cc:240] Negotiation successful
0929 10:59:31.114587 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":39}
I20260929 10:59:31.114684 31673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:31.111953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:38157 (local address 127.26.120.193:46581)
0929 10:59:31.112027 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:31.112029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:31.112041 (+ 12us) server_negotiation.cc:407] Connection header received
0929 10:59:31.112078 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:31.112079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:31.112089 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:31.112163 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:31.112338 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:31.112892 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:31.113921 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:31.114317 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:31.114372 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:31.114379 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:31.114566 (+ 187us) server_negotiation.cc:299] Negotiation successful
0929 10:59:31.114612 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 10:59:31.114938 31674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:31.112118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:35469 (local address 127.26.120.195:44105)
0929 10:59:31.112164 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:31.112166 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:31.112241 (+ 75us) server_negotiation.cc:407] Connection header received
0929 10:59:31.112300 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:31.112301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:31.112311 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:31.112367 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:31.112513 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:31.112982 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:31.113954 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:31.114566 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:31.114626 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:31.114634 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:31.114826 (+ 192us) server_negotiation.cc:299] Negotiation successful
0929 10:59:31.114870 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260929 10:59:31.115044 31349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8eefb8b3ab124b08913772a31ac88f07" candidate_uuid: "c7b8cbdb0dbb4244b897e4324123b13f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a7bde56ef664c0fbb2dc379102e8f08" is_pre_election: true
I20260929 10:59:31.115216 31349 raft_consensus.cc:2454] T 8eefb8b3ab124b08913772a31ac88f07 P 0a7bde56ef664c0fbb2dc379102e8f08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7b8cbdb0dbb4244b897e4324123b13f in term 0.
I20260929 10:59:31.115224 31684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:31.112010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:44105 (local address 127.26.120.194:35469)
0929 10:59:31.112181 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:31.112192 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:31.112253 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:31.112380 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:31.112381 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:31.112383 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:31.112445 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:31.112447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:31.113076 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:31.113077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:31.113829 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:31.113831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:31.114647 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:31.114648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:31.114726 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:31.114730 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:59:31.115068 (+ 338us) client_negotiation.cc:240] Negotiation successful
0929 10:59:31.115121 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":1,"thread_start_us":103,"threads_started":1}
I20260929 10:59:31.115479 31416 leader_election.cc:302] T 8eefb8b3ab124b08913772a31ac88f07 P c7b8cbdb0dbb4244b897e4324123b13f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a7bde56ef664c0fbb2dc379102e8f08, c7b8cbdb0dbb4244b897e4324123b13f; no voters:
I20260929 10:59:31.115556 31615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8eefb8b3ab124b08913772a31ac88f07" candidate_uuid: "c7b8cbdb0dbb4244b897e4324123b13f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd9f3615c1574fffbc439f5f55f6de7e" is_pre_election: true
I20260929 10:59:31.115712 31615 raft_consensus.cc:2454] T 8eefb8b3ab124b08913772a31ac88f07 P bd9f3615c1574fffbc439f5f55f6de7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7b8cbdb0dbb4244b897e4324123b13f in term 0.
I20260929 10:59:31.115684 31681 raft_consensus.cc:2781] T 8eefb8b3ab124b08913772a31ac88f07 P c7b8cbdb0dbb4244b897e4324123b13f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:59:31.115753 31681 raft_consensus.cc:482] T 8eefb8b3ab124b08913772a31ac88f07 P c7b8cbdb0dbb4244b897e4324123b13f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:59:31.115783 31681 raft_consensus.cc:3037] T 8eefb8b3ab124b08913772a31ac88f07 P c7b8cbdb0dbb4244b897e4324123b13f [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:31.116474 31681 raft_consensus.cc:504] T 8eefb8b3ab124b08913772a31ac88f07 P c7b8cbdb0dbb4244b897e4324123b13f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7b8cbdb0dbb4244b897e4324123b13f" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 42997 } } peers { permanent_uuid: "0a7bde56ef664c0fbb2dc379102e8f08" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46581 } } peers { permanent_uuid: "bd9f3615c1574fffbc439f5f55f6de7e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44105 } }
I20260929 10:59:31.116611 31681 leader_election.cc:288] T 8eefb8b3ab124b08913772a31ac88f07 P c7b8cbdb0dbb4244b897e4324123b13f [CANDIDATE]: Term 1 election: Requested vote from peers 0a7bde56ef664c0fbb2dc379102e8f08 (127.26.120.193:46581), bd9f3615c1574fffbc439f5f55f6de7e (127.26.120.195:44105)
I20260929 10:59:31.116760 31349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8eefb8b3ab124b08913772a31ac88f07" candidate_uuid: "c7b8cbdb0dbb4244b897e4324123b13f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a7bde56ef664c0fbb2dc379102e8f08"
I20260929 10:59:31.116822 31615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8eefb8b3ab124b08913772a31ac88f07" candidate_uuid: "c7b8cbdb0dbb4244b897e4324123b13f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd9f3615c1574fffbc439f5f55f6de7e"
I20260929 10:59:31.116833 31349 raft_consensus.cc:3037] T 8eefb8b3ab124b08913772a31ac88f07 P 0a7bde56ef664c0fbb2dc379102e8f08 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:31.116904 31615 raft_consensus.cc:3037] T 8eefb8b3ab124b08913772a31ac88f07 P bd9f3615c1574fffbc439f5f55f6de7e [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:31.117578 31349 raft_consensus.cc:2454] T 8eefb8b3ab124b08913772a31ac88f07 P 0a7bde56ef664c0fbb2dc379102e8f08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7b8cbdb0dbb4244b897e4324123b13f in term 1.
I20260929 10:59:31.117677 31615 raft_consensus.cc:2454] T 8eefb8b3ab124b08913772a31ac88f07 P bd9f3615c1574fffbc439f5f55f6de7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7b8cbdb0dbb4244b897e4324123b13f in term 1.
I20260929 10:59:31.117738 31416 leader_election.cc:302] T 8eefb8b3ab124b08913772a31ac88f07 P c7b8cbdb0dbb4244b897e4324123b13f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a7bde56ef664c0fbb2dc379102e8f08, c7b8cbdb0dbb4244b897e4324123b13f; no voters:
I20260929 10:59:31.117871 31681 raft_consensus.cc:2781] T 8eefb8b3ab124b08913772a31ac88f07 P c7b8cbdb0dbb4244b897e4324123b13f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:59:31.118046 31681 raft_consensus.cc:686] T 8eefb8b3ab124b08913772a31ac88f07 P c7b8cbdb0dbb4244b897e4324123b13f [term 1 LEADER]: Becoming Leader. State: Replica: c7b8cbdb0dbb4244b897e4324123b13f, State: Running, Role: LEADER
I20260929 10:59:31.118170 31681 consensus_queue.cc:237] T 8eefb8b3ab124b08913772a31ac88f07 P c7b8cbdb0dbb4244b897e4324123b13f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7b8cbdb0dbb4244b897e4324123b13f" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 42997 } } peers { permanent_uuid: "0a7bde56ef664c0fbb2dc379102e8f08" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46581 } } peers { permanent_uuid: "bd9f3615c1574fffbc439f5f55f6de7e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44105 } }
I20260929 10:59:31.118966 31204 catalog_manager.cc:5495] T 8eefb8b3ab124b08913772a31ac88f07 P c7b8cbdb0dbb4244b897e4324123b13f reported cstate change: term changed from 0 to 1, leader changed from <none> to c7b8cbdb0dbb4244b897e4324123b13f (127.26.120.194). New cstate: current_term: 1 leader_uuid: "c7b8cbdb0dbb4244b897e4324123b13f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7b8cbdb0dbb4244b897e4324123b13f" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 42997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a7bde56ef664c0fbb2dc379102e8f08" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd9f3615c1574fffbc439f5f55f6de7e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44105 } health_report { overall_health: UNKNOWN } } }
I20260929 10:59:31.133675 31671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:31.131501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60356 (local address 127.26.120.194:42997)
0929 10:59:31.131565 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:31.131567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:31.131580 (+ 13us) server_negotiation.cc:407] Connection header received
0929 10:59:31.131613 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:31.131613 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:31.131630 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:31.131691 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:59:31.131803 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:31.132271 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:31.132982 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:31.133159 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:31.133210 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:31.133342 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:59:31.133438 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:59:31.133504 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:31.133525 (+ 21us) server_negotiation.cc:299] Negotiation successful
0929 10:59:31.133540 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 10:59:31.135494 31615 raft_consensus.cc:1261] T 8eefb8b3ab124b08913772a31ac88f07 P bd9f3615c1574fffbc439f5f55f6de7e [term 1 FOLLOWER]: Refusing update from remote peer c7b8cbdb0dbb4244b897e4324123b13f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:59:31.135496 31349 raft_consensus.cc:1261] T 8eefb8b3ab124b08913772a31ac88f07 P 0a7bde56ef664c0fbb2dc379102e8f08 [term 1 FOLLOWER]: Refusing update from remote peer c7b8cbdb0dbb4244b897e4324123b13f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:59:31.135886 31685 consensus_queue.cc:1035] T 8eefb8b3ab124b08913772a31ac88f07 P c7b8cbdb0dbb4244b897e4324123b13f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a7bde56ef664c0fbb2dc379102e8f08" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:59:31.136029 31681 consensus_queue.cc:1035] T 8eefb8b3ab124b08913772a31ac88f07 P c7b8cbdb0dbb4244b897e4324123b13f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd9f3615c1574fffbc439f5f55f6de7e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 44105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:59:31.138504 31688 mvcc.cc:205] Tried to move back new op lower bound from 7334623523368251392 to 7334623523300532224. Current Snapshot: MvccSnapshot[applied={T|T < 7334623523368251392}]
I20260929 10:59:31.138644 31689 mvcc.cc:205] Tried to move back new op lower bound from 7334623523368251392 to 7334623523300532224. Current Snapshot: MvccSnapshot[applied={T|T < 7334623523368251392}]
I20260929 10:59:31.140627 31204 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:59:31.140718 31204 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:59:31.141503 31204 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 42b1e8f08f7a4d22a4f13b706add932e: Sending DeleteTablet for 3 replicas of tablet 8eefb8b3ab124b08913772a31ac88f07
I20260929 10:59:31.141799 31328 tablet_service.cc:1514] Processing DeleteTablet for tablet 8eefb8b3ab124b08913772a31ac88f07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:31 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:36438
I20260929 10:59:31.141834 31595 tablet_service.cc:1514] Processing DeleteTablet for tablet 8eefb8b3ab124b08913772a31ac88f07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:31 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:47900
I20260929 10:59:31.141867 31462 tablet_service.cc:1514] Processing DeleteTablet for tablet 8eefb8b3ab124b08913772a31ac88f07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:31 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:60352
I20260929 10:59:31.142120 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 31266
I20260929 10:59:31.146203 31700 tablet_replica.cc:330] T 8eefb8b3ab124b08913772a31ac88f07 P c7b8cbdb0dbb4244b897e4324123b13f: stopping tablet replica
I20260929 10:59:31.146203 31699 tablet_replica.cc:330] T 8eefb8b3ab124b08913772a31ac88f07 P bd9f3615c1574fffbc439f5f55f6de7e: stopping tablet replica
I20260929 10:59:31.146356 31699 raft_consensus.cc:2229] T 8eefb8b3ab124b08913772a31ac88f07 P bd9f3615c1574fffbc439f5f55f6de7e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:59:31.146392 31700 raft_consensus.cc:2229] T 8eefb8b3ab124b08913772a31ac88f07 P c7b8cbdb0dbb4244b897e4324123b13f [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:59:31.146476 31699 raft_consensus.cc:2258] T 8eefb8b3ab124b08913772a31ac88f07 P bd9f3615c1574fffbc439f5f55f6de7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:31.146602 31700 raft_consensus.cc:2258] T 8eefb8b3ab124b08913772a31ac88f07 P c7b8cbdb0dbb4244b897e4324123b13f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:31.146903 31699 ts_tablet_manager.cc:1905] T 8eefb8b3ab124b08913772a31ac88f07 P bd9f3615c1574fffbc439f5f55f6de7e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:31.147063 31700 ts_tablet_manager.cc:1905] T 8eefb8b3ab124b08913772a31ac88f07 P c7b8cbdb0dbb4244b897e4324123b13f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260929 10:59:31.148169 31189 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.120.193:46581 (error 108)
I20260929 10:59:31.148718 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 31400
I20260929 10:59:31.148988 31700 ts_tablet_manager.cc:1918] T 8eefb8b3ab124b08913772a31ac88f07 P c7b8cbdb0dbb4244b897e4324123b13f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:59:31.149128 31700 log.cc:1196] T 8eefb8b3ab124b08913772a31ac88f07 P c7b8cbdb0dbb4244b897e4324123b13f: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-1/wal/wals/8eefb8b3ab124b08913772a31ac88f07
I20260929 10:59:31.149338 31699 ts_tablet_manager.cc:1918] T 8eefb8b3ab124b08913772a31ac88f07 P bd9f3615c1574fffbc439f5f55f6de7e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:59:31.149411 31699 log.cc:1196] T 8eefb8b3ab124b08913772a31ac88f07 P bd9f3615c1574fffbc439f5f55f6de7e: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790679549986039-27107-0/minicluster-data/ts-2/wal/wals/8eefb8b3ab124b08913772a31ac88f07
I20260929 10:59:31.149610 31670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:31.149492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:46581 (local address 127.0.0.1:36440)
0929 10:59:31.149535 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:31.149572 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.193:46581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260929 10:59:31.149684 31699 ts_tablet_manager.cc:1939] T 8eefb8b3ab124b08913772a31ac88f07 P bd9f3615c1574fffbc439f5f55f6de7e: Deleting consensus metadata
W20260929 10:59:31.149775 31189 catalog_manager.cc:4584] TS 0a7bde56ef664c0fbb2dc379102e8f08 (127.26.120.193:46581): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8eefb8b3ab124b08913772a31ac88f07: Network error: Client connection negotiation failed: client connection to 127.26.120.193:46581: connect: Connection refused (error 111)
I20260929 10:59:31.150090 31189 catalog_manager.cc:4841] TS bd9f3615c1574fffbc439f5f55f6de7e (127.26.120.195:44105): tablet 8eefb8b3ab124b08913772a31ac88f07 (table test-table [id=f8518d625ce544c3a083796bc360830d]) successfully deleted
I20260929 10:59:31.153925 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 31533
I20260929 10:59:31.154110 31670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:31.153987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:42997 (local address 127.0.0.1:60366)
0929 10:59:31.154037 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:31.154076 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.194:42997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260929 10:59:31.158413 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 31172
2026-09-29T10:59:31Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1970 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-29T10:59:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:59:31Z Disabled control of system clock
I20260929 10:59:31.185824 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:40591
--webserver_interface=127.26.120.254
--webserver_port=0
--builtin_ntp_servers=127.26.120.212:37613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:40591
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260929 10:59:31.265154 31705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:31.265349 31705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:31.265369 31705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:31.266420 31705 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:59:31.266451 31705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:31.266464 31705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:31.266476 31705 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:59:31.266489 31705 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:59:31.267707 31705 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:37613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:40591
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:40591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:31.267910 31705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:31.268117 31705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:31.272964 31711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:31.272953 31705 server_base.cc:1034] running on GCE node
W20260929 10:59:31.273068 31713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:31.272953 31710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:31.273386 31705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:31.273638 31705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:31.274776 31705 hybrid_clock.cc:648] HybridClock initialized: now 1790679571274760 us; error 34 us; skew 500 ppm
I20260929 10:59:31.275921 31705 webserver.cc:460] Webserver started at http://127.26.120.254:45703/ using document root <none> and password file <none>
I20260929 10:59:31.276116 31705 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:31.276157 31705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:31.276247 31705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:31.277201 31705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "e07daf8b0b8643fd8ada4d28ce5dccf5"
format_stamp: "Formatted at 2026-09-29 10:59:31 on dist-test-slave-2lzq"
server_key: "48f9c48b6fdae18b951b64c37f45a782"
server_key_iv: "299b89af8650555e99fb7313e8f04b53"
server_key_version: "encryptionkey@0"
I20260929 10:59:31.277516 31705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "e07daf8b0b8643fd8ada4d28ce5dccf5"
format_stamp: "Formatted at 2026-09-29 10:59:31 on dist-test-slave-2lzq"
server_key: "48f9c48b6fdae18b951b64c37f45a782"
server_key_iv: "299b89af8650555e99fb7313e8f04b53"
server_key_version: "encryptionkey@0"
I20260929 10:59:31.278855 31705 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:59:31.279567 31719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:31.279757 31705 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:59:31.279829 31705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "e07daf8b0b8643fd8ada4d28ce5dccf5"
format_stamp: "Formatted at 2026-09-29 10:59:31 on dist-test-slave-2lzq"
server_key: "48f9c48b6fdae18b951b64c37f45a782"
server_key_iv: "299b89af8650555e99fb7313e8f04b53"
server_key_version: "encryptionkey@0"
I20260929 10:59:31.279918 31705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:31.283141 31705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:31.283397 31705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:31.283535 31705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:31.287245 31705 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:40591
I20260929 10:59:31.287272 31771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:40591 every 8 connection(s)
I20260929 10:59:31.287592 31705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 10:59:31.288220 31772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:31.289834 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 31705
I20260929 10:59:31.289940 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 10:59:31.290182 27107 external_mini_cluster.cc:1428] Setting key 62d3eea145f0cba1bf314ee9556f8da8
I20260929 10:59:31.291118 31772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e07daf8b0b8643fd8ada4d28ce5dccf5: Bootstrap starting.
I20260929 10:59:31.291673 31772 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e07daf8b0b8643fd8ada4d28ce5dccf5: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:31.292037 31772 log.cc:824] T 00000000000000000000000000000000 P e07daf8b0b8643fd8ada4d28ce5dccf5: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:31.292804 31772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e07daf8b0b8643fd8ada4d28ce5dccf5: No bootstrap required, opened a new log
I20260929 10:59:31.292858 31775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:31.290740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46786 (local address 127.26.120.254:40591)
0929 10:59:31.290956 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:31.290963 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:31.290990 (+ 27us) server_negotiation.cc:407] Connection header received
0929 10:59:31.291452 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:31.291459 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:31.291510 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:31.291661 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:31.292379 (+ 718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:31.292382 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:31.292383 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:59:31.292386 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:31.292422 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:59:31.292431 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:31.292543 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:31.292617 (+ 74us) server_negotiation.cc:299] Negotiation successful
0929 10:59:31.292672 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":99,"threads_started":1}
I20260929 10:59:31.294363 31772 raft_consensus.cc:348] T 00000000000000000000000000000000 P e07daf8b0b8643fd8ada4d28ce5dccf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07daf8b0b8643fd8ada4d28ce5dccf5" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 40591 } }
I20260929 10:59:31.294492 31772 raft_consensus.cc:374] T 00000000000000000000000000000000 P e07daf8b0b8643fd8ada4d28ce5dccf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:31.294515 31772 raft_consensus.cc:729] T 00000000000000000000000000000000 P e07daf8b0b8643fd8ada4d28ce5dccf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e07daf8b0b8643fd8ada4d28ce5dccf5, State: Initialized, Role: FOLLOWER
I20260929 10:59:31.294629 31772 consensus_queue.cc:260] T 00000000000000000000000000000000 P e07daf8b0b8643fd8ada4d28ce5dccf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07daf8b0b8643fd8ada4d28ce5dccf5" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 40591 } }
I20260929 10:59:31.294956 31772 raft_consensus.cc:388] T 00000000000000000000000000000000 P e07daf8b0b8643fd8ada4d28ce5dccf5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:59:31.294981 31772 raft_consensus.cc:482] T 00000000000000000000000000000000 P e07daf8b0b8643fd8ada4d28ce5dccf5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:59:31.295014 31772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e07daf8b0b8643fd8ada4d28ce5dccf5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:31.295491 31772 raft_consensus.cc:504] T 00000000000000000000000000000000 P e07daf8b0b8643fd8ada4d28ce5dccf5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07daf8b0b8643fd8ada4d28ce5dccf5" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 40591 } }
I20260929 10:59:31.295583 31772 leader_election.cc:302] T 00000000000000000000000000000000 P e07daf8b0b8643fd8ada4d28ce5dccf5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e07daf8b0b8643fd8ada4d28ce5dccf5; no voters:
I20260929 10:59:31.295706 31772 leader_election.cc:288] T 00000000000000000000000000000000 P e07daf8b0b8643fd8ada4d28ce5dccf5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:59:31.295768 31777 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e07daf8b0b8643fd8ada4d28ce5dccf5 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:59:31.295910 31777 raft_consensus.cc:686] T 00000000000000000000000000000000 P e07daf8b0b8643fd8ada4d28ce5dccf5 [term 1 LEADER]: Becoming Leader. State: Replica: e07daf8b0b8643fd8ada4d28ce5dccf5, State: Running, Role: LEADER
I20260929 10:59:31.295979 31772 sys_catalog.cc:564] T 00000000000000000000000000000000 P e07daf8b0b8643fd8ada4d28ce5dccf5 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:59:31.296088 31777 consensus_queue.cc:237] T 00000000000000000000000000000000 P e07daf8b0b8643fd8ada4d28ce5dccf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07daf8b0b8643fd8ada4d28ce5dccf5" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 40591 } }
I20260929 10:59:31.296500 31777 sys_catalog.cc:455] T 00000000000000000000000000000000 P e07daf8b0b8643fd8ada4d28ce5dccf5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e07daf8b0b8643fd8ada4d28ce5dccf5. Latest consensus state: current_term: 1 leader_uuid: "e07daf8b0b8643fd8ada4d28ce5dccf5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07daf8b0b8643fd8ada4d28ce5dccf5" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 40591 } } }
I20260929 10:59:31.296586 31777 sys_catalog.cc:458] T 00000000000000000000000000000000 P e07daf8b0b8643fd8ada4d28ce5dccf5 [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:31.296490 31778 sys_catalog.cc:455] T 00000000000000000000000000000000 P e07daf8b0b8643fd8ada4d28ce5dccf5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e07daf8b0b8643fd8ada4d28ce5dccf5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07daf8b0b8643fd8ada4d28ce5dccf5" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 40591 } } }
I20260929 10:59:31.296790 31778 sys_catalog.cc:458] T 00000000000000000000000000000000 P e07daf8b0b8643fd8ada4d28ce5dccf5 [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:31.296941 31784 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:59:31.297508 31784 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:59:31.302532 31784 catalog_manager.cc:1347] Generated new cluster ID: 3840e2bd59784e579a1808217e798515
I20260929 10:59:31.302588 31784 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:59:31.318035 31784 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:59:31.318260 31784 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:59:31.321679 31784 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e07daf8b0b8643fd8ada4d28ce5dccf5: Generated new TSK 0
I20260929 10:59:31.321861 31784 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:59:31.330536 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.193:0
--local_ip_for_outbound_sockets=127.26.120.193
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:40591
--builtin_ntp_servers=127.26.120.212:37613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260929 10:59:31.409829 31796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:31.410102 31796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:31.410154 31796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:31.411307 31796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:31.411345 31796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:31.411369 31796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.193
I20260929 10:59:31.412731 31796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:37613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:40591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.193
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:31.412977 31796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:31.413281 31796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:31.418286 31804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:31.418313 31802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:31.418316 31801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:31.418547 31796 server_base.cc:1034] running on GCE node
I20260929 10:59:31.418809 31796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:31.419071 31796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:31.420226 31796 hybrid_clock.cc:648] HybridClock initialized: now 1790679571420216 us; error 26 us; skew 500 ppm
I20260929 10:59:31.421562 31796 webserver.cc:460] Webserver started at http://127.26.120.193:39245/ using document root <none> and password file <none>
I20260929 10:59:31.421809 31796 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:31.421862 31796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:31.421990 31796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:31.423321 31796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-0/data/instance:
uuid: "aa4643e48d204a2488ef3309fe7bc6d0"
format_stamp: "Formatted at 2026-09-29 10:59:31 on dist-test-slave-2lzq"
server_key: "b755e4eb89371141f8d307766c096795"
server_key_iv: "27f8e240db1a545a345e0d669d5d459b"
server_key_version: "encryptionkey@0"
I20260929 10:59:31.423753 31796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance:
uuid: "aa4643e48d204a2488ef3309fe7bc6d0"
format_stamp: "Formatted at 2026-09-29 10:59:31 on dist-test-slave-2lzq"
server_key: "b755e4eb89371141f8d307766c096795"
server_key_iv: "27f8e240db1a545a345e0d669d5d459b"
server_key_version: "encryptionkey@0"
I20260929 10:59:31.425343 31796 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:59:31.426152 31810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:31.426352 31796 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:59:31.426437 31796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-0/wal
uuid: "aa4643e48d204a2488ef3309fe7bc6d0"
format_stamp: "Formatted at 2026-09-29 10:59:31 on dist-test-slave-2lzq"
server_key: "b755e4eb89371141f8d307766c096795"
server_key_iv: "27f8e240db1a545a345e0d669d5d459b"
server_key_version: "encryptionkey@0"
I20260929 10:59:31.426508 31796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:31.430101 31796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:31.430333 31796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:31.430428 31796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:31.430580 31796 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:59:31.430835 31796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:31.430871 31796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:31.430891 31796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:31.430903 31796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:31.436512 31796 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.193:42647
I20260929 10:59:31.436580 31923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.193:42647 every 8 connection(s)
I20260929 10:59:31.436880 31796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
I20260929 10:59:31.439965 31775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:31.437622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:49377 (local address 127.26.120.254:40591)
0929 10:59:31.437684 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:31.437687 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:31.437905 (+ 218us) server_negotiation.cc:407] Connection header received
0929 10:59:31.438449 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:31.438451 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:31.438462 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:31.438517 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:31.439721 (+ 1204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:31.439722 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:31.439722 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:59:31.439727 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:31.439741 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:59:31.439745 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:31.439792 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:31.439882 (+ 90us) server_negotiation.cc:299] Negotiation successful
0929 10:59:31.439896 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 10:59:31.440164 31926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:31.437626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:40591 (local address 127.26.120.193:49377)
0929 10:59:31.437791 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:31.437823 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:31.438346 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:31.438595 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:31.438599 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:31.438608 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:31.439563 (+ 955us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:59:31.439563 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:31.439590 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:59:31.439617 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:31.439791 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:31.439796 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:31.439799 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:59:31.439892 (+ 93us) client_negotiation.cc:240] Negotiation successful
0929 10:59:31.439952 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":75,"threads_started":1}
I20260929 10:59:31.440667 31924 heartbeater.cc:347] Connected to a master server at 127.26.120.254:40591
I20260929 10:59:31.440739 31924 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:31.440897 31924 heartbeater.cc:511] Master 127.26.120.254:40591 requested a full tablet report, sending...
I20260929 10:59:31.441409 31736 ts_manager.cc:194] Registered new tserver with Master: aa4643e48d204a2488ef3309fe7bc6d0 (127.26.120.193:42647)
I20260929 10:59:31.445413 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 31796
I20260929 10:59:31.445518 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance
I20260929 10:59:31.445695 27107 external_mini_cluster.cc:1428] Setting key 9d7fcec1a31d3b6bd2f92d5c46234dbf
I20260929 10:59:31.446645 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.194:0
--local_ip_for_outbound_sockets=127.26.120.194
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:40591
--builtin_ntp_servers=127.26.120.212:37613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260929 10:59:31.526372 31927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:31.526580 31927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:31.526603 31927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:31.527767 31927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:31.527817 31927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:31.527849 31927 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.194
I20260929 10:59:31.529531 31927 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:37613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:40591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.194
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:31.529738 31927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:31.529991 31927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 10:59:31.535198 31927 server_base.cc:1034] running on GCE node
W20260929 10:59:31.535313 31935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:31.535353 31933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:31.535395 31932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:31.535692 31927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:31.535902 31927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:31.537039 31927 hybrid_clock.cc:648] HybridClock initialized: now 1790679571537017 us; error 35 us; skew 500 ppm
I20260929 10:59:31.538388 31927 webserver.cc:460] Webserver started at http://127.26.120.194:46489/ using document root <none> and password file <none>
I20260929 10:59:31.538642 31927 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:31.538690 31927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:31.538861 31927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:31.540002 31927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-1/data/instance:
uuid: "e744377d7a2c4927b9ce291768ea8c87"
format_stamp: "Formatted at 2026-09-29 10:59:31 on dist-test-slave-2lzq"
server_key: "71938579e82614d8cf861225b55f532a"
server_key_iv: "967da4abbe8e9af1a7a4c2bd423d79a8"
server_key_version: "encryptionkey@0"
I20260929 10:59:31.540362 31927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance:
uuid: "e744377d7a2c4927b9ce291768ea8c87"
format_stamp: "Formatted at 2026-09-29 10:59:31 on dist-test-slave-2lzq"
server_key: "71938579e82614d8cf861225b55f532a"
server_key_iv: "967da4abbe8e9af1a7a4c2bd423d79a8"
server_key_version: "encryptionkey@0"
I20260929 10:59:31.541689 31927 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 10:59:31.542472 31941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:31.542658 31927 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:31.542763 31927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-1/wal
uuid: "e744377d7a2c4927b9ce291768ea8c87"
format_stamp: "Formatted at 2026-09-29 10:59:31 on dist-test-slave-2lzq"
server_key: "71938579e82614d8cf861225b55f532a"
server_key_iv: "967da4abbe8e9af1a7a4c2bd423d79a8"
server_key_version: "encryptionkey@0"
I20260929 10:59:31.542841 31927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:31.546278 31927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:31.546574 31927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:31.546723 31927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:31.546928 31927 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:59:31.547272 31927 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:31.547335 31927 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:31.547377 31927 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:31.547405 31927 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:31.553272 31927 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.194:34531
I20260929 10:59:31.553321 32054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.194:34531 every 8 connection(s)
I20260929 10:59:31.553666 31927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
I20260929 10:59:31.556730 31775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:31.554413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:46627 (local address 127.26.120.254:40591)
0929 10:59:31.554458 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:31.554460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:31.554759 (+ 299us) server_negotiation.cc:407] Connection header received
0929 10:59:31.555283 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:31.555284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:31.555291 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:31.555342 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:31.556476 (+ 1134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:31.556477 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:31.556478 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:59:31.556483 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:31.556500 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:59:31.556504 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:31.556557 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:31.556642 (+ 85us) server_negotiation.cc:299] Negotiation successful
0929 10:59:31.556659 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 10:59:31.556922 32057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:31.554423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:40591 (local address 127.26.120.194:46627)
0929 10:59:31.554628 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:31.554663 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:31.555163 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:31.555411 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:31.555415 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:31.555425 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:31.556315 (+ 890us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:59:31.556316 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:31.556344 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:59:31.556375 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:31.556554 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:31.556559 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:31.556562 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:59:31.556637 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 10:59:31.556703 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":95,"threads_started":1}
I20260929 10:59:31.557502 32055 heartbeater.cc:347] Connected to a master server at 127.26.120.254:40591
I20260929 10:59:31.557622 32055 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:31.557782 32055 heartbeater.cc:511] Master 127.26.120.254:40591 requested a full tablet report, sending...
I20260929 10:59:31.558236 31736 ts_manager.cc:194] Registered new tserver with Master: e744377d7a2c4927b9ce291768ea8c87 (127.26.120.194:34531)
I20260929 10:59:31.561095 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 31927
I20260929 10:59:31.561204 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance
I20260929 10:59:31.561378 27107 external_mini_cluster.cc:1428] Setting key 5bb9af53c20c3ef2e5ac380f9f757900
I20260929 10:59:31.562335 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.195:0
--local_ip_for_outbound_sockets=127.26.120.195
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:40591
--builtin_ntp_servers=127.26.120.212:37613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260929 10:59:31.642143 32058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:31.642334 32058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:31.642355 32058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:31.643517 32058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:31.643564 32058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:31.643625 32058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.195
I20260929 10:59:31.644927 32058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:37613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:40591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.195
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:31.645187 32058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:31.645542 32058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:31.650696 32064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:31.650743 32066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:31.650743 32063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:31.650800 32058 server_base.cc:1034] running on GCE node
I20260929 10:59:31.651110 32058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:31.651351 32058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:31.652529 32058 hybrid_clock.cc:648] HybridClock initialized: now 1790679571652515 us; error 29 us; skew 500 ppm
I20260929 10:59:31.653762 32058 webserver.cc:460] Webserver started at http://127.26.120.195:40593/ using document root <none> and password file <none>
I20260929 10:59:31.654078 32058 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:31.654145 32058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:31.654282 32058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:31.655364 32058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-2/data/instance:
uuid: "dfa54eee186e4771a6f1c52379ee9f8f"
format_stamp: "Formatted at 2026-09-29 10:59:31 on dist-test-slave-2lzq"
server_key: "f6095422b6edfc70414222aba8355e6b"
server_key_iv: "ef34982d91faad7ec7901694de45e94d"
server_key_version: "encryptionkey@0"
I20260929 10:59:31.655740 32058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance:
uuid: "dfa54eee186e4771a6f1c52379ee9f8f"
format_stamp: "Formatted at 2026-09-29 10:59:31 on dist-test-slave-2lzq"
server_key: "f6095422b6edfc70414222aba8355e6b"
server_key_iv: "ef34982d91faad7ec7901694de45e94d"
server_key_version: "encryptionkey@0"
I20260929 10:59:31.657042 32058 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:59:31.657788 32072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:31.657963 32058 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:59:31.658059 32058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-2/wal
uuid: "dfa54eee186e4771a6f1c52379ee9f8f"
format_stamp: "Formatted at 2026-09-29 10:59:31 on dist-test-slave-2lzq"
server_key: "f6095422b6edfc70414222aba8355e6b"
server_key_iv: "ef34982d91faad7ec7901694de45e94d"
server_key_version: "encryptionkey@0"
I20260929 10:59:31.658175 32058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:31.661427 32058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:31.661697 32058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:31.661830 32058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:31.662035 32058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:59:31.662366 32058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:31.662420 32058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:31.662457 32058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:31.662483 32058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:31.668321 32058 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.195:45035
I20260929 10:59:31.668391 32185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.195:45035 every 8 connection(s)
I20260929 10:59:31.668706 32058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
I20260929 10:59:31.671772 31775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:31.669430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:40669 (local address 127.26.120.254:40591)
0929 10:59:31.669527 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:31.669529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:31.669760 (+ 231us) server_negotiation.cc:407] Connection header received
0929 10:59:31.670351 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:31.670352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:31.670358 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:31.670414 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:31.671529 (+ 1115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:31.671530 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:31.671530 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:59:31.671535 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:31.671549 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:59:31.671553 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:31.671604 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:31.671688 (+ 84us) server_negotiation.cc:299] Negotiation successful
0929 10:59:31.671704 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 10:59:31.671955 32188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:31.669389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:40591 (local address 127.26.120.195:40669)
0929 10:59:31.669622 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:31.669663 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:31.670242 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:31.670485 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:31.670489 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:31.670498 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:31.671376 (+ 878us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:59:31.671376 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:31.671404 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:59:31.671429 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:31.671616 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:31.671620 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:31.671623 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:59:31.671688 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 10:59:31.671748 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":83,"threads_started":1}
I20260929 10:59:31.672407 32186 heartbeater.cc:347] Connected to a master server at 127.26.120.254:40591
I20260929 10:59:31.672479 32186 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:31.672652 32186 heartbeater.cc:511] Master 127.26.120.254:40591 requested a full tablet report, sending...
I20260929 10:59:31.672999 31736 ts_manager.cc:194] Registered new tserver with Master: dfa54eee186e4771a6f1c52379ee9f8f (127.26.120.195:45035)
I20260929 10:59:31.676839 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 32058
I20260929 10:59:31.676934 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance
I20260929 10:59:31.677124 27107 external_mini_cluster.cc:1428] Setting key dc237e089cc7d65a6b680881821f7441
I20260929 10:59:31.678128 27107 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:59:31.680306 31775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:31.679378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46794 (local address 127.26.120.254:40591)
0929 10:59:31.679440 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:31.679441 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:31.679568 (+ 127us) server_negotiation.cc:407] Connection header received
0929 10:59:31.679609 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:31.679610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:31.679614 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:31.679670 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:31.680151 (+ 481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:31.680152 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:31.680152 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:59:31.680156 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:31.680168 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:59:31.680171 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:31.680215 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:31.680246 (+ 31us) server_negotiation.cc:299] Negotiation successful
0929 10:59:31.680257 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 10:59:31.681759 31736 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46794:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:59:31.683524 31736 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:59:31.686995 32197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:31.685648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42312 (local address 127.26.120.193:42647)
0929 10:59:31.685946 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:31.685951 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:31.685961 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:59:31.685987 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:31.685989 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:31.686000 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:31.686085 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:31.686675 (+ 590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:31.686683 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:31.686683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:59:31.686687 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:31.686719 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:59:31.686740 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:31.686821 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:31.686910 (+ 89us) server_negotiation.cc:299] Negotiation successful
0929 10:59:31.686926 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":76,"threads_started":1}
I20260929 10:59:31.687000 32194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:31.685566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:42647 (local address 127.0.0.1:42312)
0929 10:59:31.685739 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:31.685775 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:31.685825 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:31.686087 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:31.686092 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:31.686099 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:31.686551 (+ 452us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:59:31.686552 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:31.686569 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:59:31.686591 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:31.686823 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:31.686829 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:31.686831 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 10:59:31.686894 (+ 63us) client_negotiation.cc:240] Negotiation successful
0929 10:59:31.686908 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":69,"threads_started":1}
I20260929 10:59:31.687461 32198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:31.685567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:45035 (local address 127.0.0.1:59462)
0929 10:59:31.685907 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:31.685918 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:31.685957 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:31.686472 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:31.686473 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:31.686476 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:31.687099 (+ 623us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:59:31.687100 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:31.687105 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:59:31.687118 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:31.687332 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:31.687332 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:31.687333 (+ 1us) client_negotiation.cc:769] Sending connection context
0929 10:59:31.687389 (+ 56us) client_negotiation.cc:240] Negotiation successful
0929 10:59:31.687402 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":5,"thread_start_us":85,"threads_started":1}
I20260929 10:59:31.687487 32196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:31.685616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59462 (local address 127.26.120.195:45035)
0929 10:59:31.686023 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:31.686028 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:31.686039 (+ 11us) server_negotiation.cc:407] Connection header received
0929 10:59:31.686095 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:31.686098 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:31.686108 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:31.686190 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:31.687207 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:31.687215 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:31.687216 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:59:31.687220 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:31.687253 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:59:31.687280 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:31.687340 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:31.687398 (+ 58us) server_negotiation.cc:299] Negotiation successful
0929 10:59:31.687412 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":79,"threads_started":1}
I20260929 10:59:31.687878 32195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:31.685776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:34531 (local address 127.0.0.1:48456)
0929 10:59:31.686666 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:31.686675 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:31.686721 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:31.687144 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:31.687145 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:31.687147 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:31.687573 (+ 426us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:59:31.687573 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:31.687579 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:59:31.687590 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:31.687802 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:31.687803 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:31.687803 (+ 0us) client_negotiation.cc:769] Sending connection context
0929 10:59:31.687831 (+ 28us) client_negotiation.cc:240] Negotiation successful
0929 10:59:31.687840 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":875,"thread_start_us":67,"threads_started":1}
I20260929 10:59:31.687930 32199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:31.685968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48456 (local address 127.26.120.194:34531)
0929 10:59:31.686198 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:31.686201 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:31.686747 (+ 546us) server_negotiation.cc:407] Connection header received
0929 10:59:31.686779 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:31.686782 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:31.686792 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:31.686862 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:31.687658 (+ 796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:31.687665 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:31.687665 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:59:31.687669 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:31.687700 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:59:31.687737 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:31.687815 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:31.687839 (+ 24us) server_negotiation.cc:299] Negotiation successful
0929 10:59:31.687854 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":122,"threads_started":1}
I20260929 10:59:31.688153 31858 tablet_service.cc:1467] Processing CreateTablet for tablet 84d30203be3f44ba8ef0574d5847b979 (DEFAULT_TABLE table=test-table [id=636ad6849b884cdd87436e247deaba4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:31.688529 31858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84d30203be3f44ba8ef0574d5847b979. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:31.690819 32120 tablet_service.cc:1467] Processing CreateTablet for tablet 84d30203be3f44ba8ef0574d5847b979 (DEFAULT_TABLE table=test-table [id=636ad6849b884cdd87436e247deaba4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:31.690856 31989 tablet_service.cc:1467] Processing CreateTablet for tablet 84d30203be3f44ba8ef0574d5847b979 (DEFAULT_TABLE table=test-table [id=636ad6849b884cdd87436e247deaba4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:31.691138 32120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84d30203be3f44ba8ef0574d5847b979. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:31.691206 31989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84d30203be3f44ba8ef0574d5847b979. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:31.691378 32200 tablet_bootstrap.cc:492] T 84d30203be3f44ba8ef0574d5847b979 P aa4643e48d204a2488ef3309fe7bc6d0: Bootstrap starting.
I20260929 10:59:31.692052 32200 tablet_bootstrap.cc:654] T 84d30203be3f44ba8ef0574d5847b979 P aa4643e48d204a2488ef3309fe7bc6d0: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:31.692431 32200 log.cc:824] T 84d30203be3f44ba8ef0574d5847b979 P aa4643e48d204a2488ef3309fe7bc6d0: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:31.693291 32200 tablet_bootstrap.cc:492] T 84d30203be3f44ba8ef0574d5847b979 P aa4643e48d204a2488ef3309fe7bc6d0: No bootstrap required, opened a new log
I20260929 10:59:31.693356 32200 ts_tablet_manager.cc:1397] T 84d30203be3f44ba8ef0574d5847b979 P aa4643e48d204a2488ef3309fe7bc6d0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:59:31.693868 32202 tablet_bootstrap.cc:492] T 84d30203be3f44ba8ef0574d5847b979 P e744377d7a2c4927b9ce291768ea8c87: Bootstrap starting.
I20260929 10:59:31.694448 32203 tablet_bootstrap.cc:492] T 84d30203be3f44ba8ef0574d5847b979 P dfa54eee186e4771a6f1c52379ee9f8f: Bootstrap starting.
I20260929 10:59:31.694509 32202 tablet_bootstrap.cc:654] T 84d30203be3f44ba8ef0574d5847b979 P e744377d7a2c4927b9ce291768ea8c87: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:31.694846 32202 log.cc:824] T 84d30203be3f44ba8ef0574d5847b979 P e744377d7a2c4927b9ce291768ea8c87: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:31.695152 32203 tablet_bootstrap.cc:654] T 84d30203be3f44ba8ef0574d5847b979 P dfa54eee186e4771a6f1c52379ee9f8f: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:31.695488 32203 log.cc:824] T 84d30203be3f44ba8ef0574d5847b979 P dfa54eee186e4771a6f1c52379ee9f8f: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:31.695679 32202 tablet_bootstrap.cc:492] T 84d30203be3f44ba8ef0574d5847b979 P e744377d7a2c4927b9ce291768ea8c87: No bootstrap required, opened a new log
I20260929 10:59:31.695755 32202 ts_tablet_manager.cc:1397] T 84d30203be3f44ba8ef0574d5847b979 P e744377d7a2c4927b9ce291768ea8c87: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:59:31.695999 32200 raft_consensus.cc:348] T 84d30203be3f44ba8ef0574d5847b979 P aa4643e48d204a2488ef3309fe7bc6d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfa54eee186e4771a6f1c52379ee9f8f" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45035 } } peers { permanent_uuid: "aa4643e48d204a2488ef3309fe7bc6d0" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42647 } } peers { permanent_uuid: "e744377d7a2c4927b9ce291768ea8c87" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 34531 } }
I20260929 10:59:31.696230 32200 raft_consensus.cc:374] T 84d30203be3f44ba8ef0574d5847b979 P aa4643e48d204a2488ef3309fe7bc6d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:31.696283 32200 raft_consensus.cc:729] T 84d30203be3f44ba8ef0574d5847b979 P aa4643e48d204a2488ef3309fe7bc6d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa4643e48d204a2488ef3309fe7bc6d0, State: Initialized, Role: FOLLOWER
I20260929 10:59:31.696331 32203 tablet_bootstrap.cc:492] T 84d30203be3f44ba8ef0574d5847b979 P dfa54eee186e4771a6f1c52379ee9f8f: No bootstrap required, opened a new log
I20260929 10:59:31.696421 32203 ts_tablet_manager.cc:1397] T 84d30203be3f44ba8ef0574d5847b979 P dfa54eee186e4771a6f1c52379ee9f8f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:59:31.696412 32200 consensus_queue.cc:260] T 84d30203be3f44ba8ef0574d5847b979 P aa4643e48d204a2488ef3309fe7bc6d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfa54eee186e4771a6f1c52379ee9f8f" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45035 } } peers { permanent_uuid: "aa4643e48d204a2488ef3309fe7bc6d0" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42647 } } peers { permanent_uuid: "e744377d7a2c4927b9ce291768ea8c87" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 34531 } }
I20260929 10:59:31.696658 32200 ts_tablet_manager.cc:1428] T 84d30203be3f44ba8ef0574d5847b979 P aa4643e48d204a2488ef3309fe7bc6d0: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260929 10:59:31.696805 31924 heartbeater.cc:503] Master 127.26.120.254:40591 was elected leader, sending a full tablet report...
I20260929 10:59:31.698277 32202 raft_consensus.cc:348] T 84d30203be3f44ba8ef0574d5847b979 P e744377d7a2c4927b9ce291768ea8c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfa54eee186e4771a6f1c52379ee9f8f" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45035 } } peers { permanent_uuid: "aa4643e48d204a2488ef3309fe7bc6d0" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42647 } } peers { permanent_uuid: "e744377d7a2c4927b9ce291768ea8c87" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 34531 } }
I20260929 10:59:31.698442 32202 raft_consensus.cc:374] T 84d30203be3f44ba8ef0574d5847b979 P e744377d7a2c4927b9ce291768ea8c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:31.698480 32202 raft_consensus.cc:729] T 84d30203be3f44ba8ef0574d5847b979 P e744377d7a2c4927b9ce291768ea8c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e744377d7a2c4927b9ce291768ea8c87, State: Initialized, Role: FOLLOWER
I20260929 10:59:31.698635 32202 consensus_queue.cc:260] T 84d30203be3f44ba8ef0574d5847b979 P e744377d7a2c4927b9ce291768ea8c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfa54eee186e4771a6f1c52379ee9f8f" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45035 } } peers { permanent_uuid: "aa4643e48d204a2488ef3309fe7bc6d0" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42647 } } peers { permanent_uuid: "e744377d7a2c4927b9ce291768ea8c87" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 34531 } }
I20260929 10:59:31.698926 32055 heartbeater.cc:503] Master 127.26.120.254:40591 was elected leader, sending a full tablet report...
I20260929 10:59:31.698925 32202 ts_tablet_manager.cc:1428] T 84d30203be3f44ba8ef0574d5847b979 P e744377d7a2c4927b9ce291768ea8c87: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 10:59:31.699070 32203 raft_consensus.cc:348] T 84d30203be3f44ba8ef0574d5847b979 P dfa54eee186e4771a6f1c52379ee9f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfa54eee186e4771a6f1c52379ee9f8f" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45035 } } peers { permanent_uuid: "aa4643e48d204a2488ef3309fe7bc6d0" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42647 } } peers { permanent_uuid: "e744377d7a2c4927b9ce291768ea8c87" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 34531 } }
I20260929 10:59:31.699232 32203 raft_consensus.cc:374] T 84d30203be3f44ba8ef0574d5847b979 P dfa54eee186e4771a6f1c52379ee9f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:31.699285 32203 raft_consensus.cc:729] T 84d30203be3f44ba8ef0574d5847b979 P dfa54eee186e4771a6f1c52379ee9f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfa54eee186e4771a6f1c52379ee9f8f, State: Initialized, Role: FOLLOWER
I20260929 10:59:31.699394 32203 consensus_queue.cc:260] T 84d30203be3f44ba8ef0574d5847b979 P dfa54eee186e4771a6f1c52379ee9f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfa54eee186e4771a6f1c52379ee9f8f" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45035 } } peers { permanent_uuid: "aa4643e48d204a2488ef3309fe7bc6d0" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42647 } } peers { permanent_uuid: "e744377d7a2c4927b9ce291768ea8c87" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 34531 } }
I20260929 10:59:31.699656 32203 ts_tablet_manager.cc:1428] T 84d30203be3f44ba8ef0574d5847b979 P dfa54eee186e4771a6f1c52379ee9f8f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 10:59:31.699687 32186 heartbeater.cc:503] Master 127.26.120.254:40591 was elected leader, sending a full tablet report...
W20260929 10:59:31.804244 32056 tablet.cc:2369] T 84d30203be3f44ba8ef0574d5847b979 P e744377d7a2c4927b9ce291768ea8c87: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:59:31.817209 32207 raft_consensus.cc:482] T 84d30203be3f44ba8ef0574d5847b979 P e744377d7a2c4927b9ce291768ea8c87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:59:31.817332 32207 raft_consensus.cc:504] T 84d30203be3f44ba8ef0574d5847b979 P e744377d7a2c4927b9ce291768ea8c87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfa54eee186e4771a6f1c52379ee9f8f" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45035 } } peers { permanent_uuid: "aa4643e48d204a2488ef3309fe7bc6d0" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42647 } } peers { permanent_uuid: "e744377d7a2c4927b9ce291768ea8c87" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 34531 } }
I20260929 10:59:31.817682 32207 leader_election.cc:288] T 84d30203be3f44ba8ef0574d5847b979 P e744377d7a2c4927b9ce291768ea8c87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dfa54eee186e4771a6f1c52379ee9f8f (127.26.120.195:45035), aa4643e48d204a2488ef3309fe7bc6d0 (127.26.120.193:42647)
I20260929 10:59:31.819079 32197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:31.817948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:44971 (local address 127.26.120.193:42647)
0929 10:59:31.817996 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:31.817998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:31.818011 (+ 13us) server_negotiation.cc:407] Connection header received
0929 10:59:31.818122 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:31.818124 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:31.818131 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:31.818196 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:31.818825 (+ 629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:31.818827 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:31.818828 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:59:31.818832 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:31.818872 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:59:31.818876 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:31.818941 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:31.818992 (+ 51us) server_negotiation.cc:299] Negotiation successful
0929 10:59:31.819007 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 10:59:31.819083 32057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:31.817871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:42647 (local address 127.26.120.194:44971)
0929 10:59:31.817947 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:31.817961 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:31.818003 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:31.818234 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:31.818235 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:31.818238 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:31.818725 (+ 487us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:59:31.818725 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:31.818732 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:59:31.818743 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:31.818939 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:31.818940 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:31.818941 (+ 1us) client_negotiation.cc:769] Sending connection context
0929 10:59:31.818991 (+ 50us) client_negotiation.cc:240] Negotiation successful
0929 10:59:31.819007 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260929 10:59:31.819296 32196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:31.818002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:36733 (local address 127.26.120.195:45035)
0929 10:59:31.818086 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:31.818088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:31.818224 (+ 136us) server_negotiation.cc:407] Connection header received
0929 10:59:31.818253 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:31.818255 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:31.818260 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:31.818330 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:31.819029 (+ 699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:31.819030 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:31.819031 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:59:31.819035 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:31.819049 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:59:31.819054 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:31.819126 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:31.819203 (+ 77us) server_negotiation.cc:299] Negotiation successful
0929 10:59:31.819220 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 10:59:31.819470 32209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:31.817890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:45035 (local address 127.26.120.194:36733)
0929 10:59:31.818153 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:31.818162 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:31.818205 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:31.818340 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:31.818341 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:31.818343 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:31.818921 (+ 578us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 10:59:31.818922 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:31.818929 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 10:59:31.818945 (+ 16us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:31.819121 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:31.819122 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:31.819122 (+ 0us) client_negotiation.cc:769] Sending connection context
0929 10:59:31.819379 (+ 257us) client_negotiation.cc:240] Negotiation successful
0929 10:59:31.819393 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":71,"thread_start_us":140,"threads_started":1}
I20260929 10:59:31.819437 31878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84d30203be3f44ba8ef0574d5847b979" candidate_uuid: "e744377d7a2c4927b9ce291768ea8c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa4643e48d204a2488ef3309fe7bc6d0" is_pre_election: true
I20260929 10:59:31.819618 31878 raft_consensus.cc:2454] T 84d30203be3f44ba8ef0574d5847b979 P aa4643e48d204a2488ef3309fe7bc6d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e744377d7a2c4927b9ce291768ea8c87 in term 0.
I20260929 10:59:31.819762 32140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84d30203be3f44ba8ef0574d5847b979" candidate_uuid: "e744377d7a2c4927b9ce291768ea8c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfa54eee186e4771a6f1c52379ee9f8f" is_pre_election: true
I20260929 10:59:31.819895 31942 leader_election.cc:302] T 84d30203be3f44ba8ef0574d5847b979 P e744377d7a2c4927b9ce291768ea8c87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa4643e48d204a2488ef3309fe7bc6d0, e744377d7a2c4927b9ce291768ea8c87; no voters:
I20260929 10:59:31.819922 32140 raft_consensus.cc:2454] T 84d30203be3f44ba8ef0574d5847b979 P dfa54eee186e4771a6f1c52379ee9f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e744377d7a2c4927b9ce291768ea8c87 in term 0.
I20260929 10:59:31.820012 32207 raft_consensus.cc:2781] T 84d30203be3f44ba8ef0574d5847b979 P e744377d7a2c4927b9ce291768ea8c87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:59:31.820071 32207 raft_consensus.cc:482] T 84d30203be3f44ba8ef0574d5847b979 P e744377d7a2c4927b9ce291768ea8c87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:59:31.820106 32207 raft_consensus.cc:3037] T 84d30203be3f44ba8ef0574d5847b979 P e744377d7a2c4927b9ce291768ea8c87 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:31.820809 32207 raft_consensus.cc:504] T 84d30203be3f44ba8ef0574d5847b979 P e744377d7a2c4927b9ce291768ea8c87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfa54eee186e4771a6f1c52379ee9f8f" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45035 } } peers { permanent_uuid: "aa4643e48d204a2488ef3309fe7bc6d0" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42647 } } peers { permanent_uuid: "e744377d7a2c4927b9ce291768ea8c87" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 34531 } }
I20260929 10:59:31.820958 32207 leader_election.cc:288] T 84d30203be3f44ba8ef0574d5847b979 P e744377d7a2c4927b9ce291768ea8c87 [CANDIDATE]: Term 1 election: Requested vote from peers dfa54eee186e4771a6f1c52379ee9f8f (127.26.120.195:45035), aa4643e48d204a2488ef3309fe7bc6d0 (127.26.120.193:42647)
I20260929 10:59:31.821103 31878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84d30203be3f44ba8ef0574d5847b979" candidate_uuid: "e744377d7a2c4927b9ce291768ea8c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa4643e48d204a2488ef3309fe7bc6d0"
I20260929 10:59:31.821161 32140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84d30203be3f44ba8ef0574d5847b979" candidate_uuid: "e744377d7a2c4927b9ce291768ea8c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfa54eee186e4771a6f1c52379ee9f8f"
I20260929 10:59:31.821187 31878 raft_consensus.cc:3037] T 84d30203be3f44ba8ef0574d5847b979 P aa4643e48d204a2488ef3309fe7bc6d0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:31.821265 32140 raft_consensus.cc:3037] T 84d30203be3f44ba8ef0574d5847b979 P dfa54eee186e4771a6f1c52379ee9f8f [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:31.821952 31878 raft_consensus.cc:2454] T 84d30203be3f44ba8ef0574d5847b979 P aa4643e48d204a2488ef3309fe7bc6d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e744377d7a2c4927b9ce291768ea8c87 in term 1.
I20260929 10:59:31.821954 32140 raft_consensus.cc:2454] T 84d30203be3f44ba8ef0574d5847b979 P dfa54eee186e4771a6f1c52379ee9f8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e744377d7a2c4927b9ce291768ea8c87 in term 1.
I20260929 10:59:31.822146 31942 leader_election.cc:302] T 84d30203be3f44ba8ef0574d5847b979 P e744377d7a2c4927b9ce291768ea8c87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa4643e48d204a2488ef3309fe7bc6d0, e744377d7a2c4927b9ce291768ea8c87; no voters:
I20260929 10:59:31.822268 32207 raft_consensus.cc:2781] T 84d30203be3f44ba8ef0574d5847b979 P e744377d7a2c4927b9ce291768ea8c87 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:59:31.822396 32207 raft_consensus.cc:686] T 84d30203be3f44ba8ef0574d5847b979 P e744377d7a2c4927b9ce291768ea8c87 [term 1 LEADER]: Becoming Leader. State: Replica: e744377d7a2c4927b9ce291768ea8c87, State: Running, Role: LEADER
I20260929 10:59:31.822492 32207 consensus_queue.cc:237] T 84d30203be3f44ba8ef0574d5847b979 P e744377d7a2c4927b9ce291768ea8c87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfa54eee186e4771a6f1c52379ee9f8f" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45035 } } peers { permanent_uuid: "aa4643e48d204a2488ef3309fe7bc6d0" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42647 } } peers { permanent_uuid: "e744377d7a2c4927b9ce291768ea8c87" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 34531 } }
I20260929 10:59:31.823307 31736 catalog_manager.cc:5495] T 84d30203be3f44ba8ef0574d5847b979 P e744377d7a2c4927b9ce291768ea8c87 reported cstate change: term changed from 0 to 1, leader changed from <none> to e744377d7a2c4927b9ce291768ea8c87 (127.26.120.194). New cstate: current_term: 1 leader_uuid: "e744377d7a2c4927b9ce291768ea8c87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfa54eee186e4771a6f1c52379ee9f8f" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa4643e48d204a2488ef3309fe7bc6d0" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e744377d7a2c4927b9ce291768ea8c87" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 34531 } health_report { overall_health: HEALTHY } } }
I20260929 10:59:31.833922 32199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:31.833025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48462 (local address 127.26.120.194:34531)
0929 10:59:31.833089 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:31.833090 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:31.833099 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:59:31.833131 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:31.833131 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:31.833142 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:31.833198 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:31.833730 (+ 532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:31.833731 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:31.833731 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:59:31.833735 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:31.833750 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:59:31.833754 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:31.833802 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:31.833859 (+ 57us) server_negotiation.cc:299] Negotiation successful
0929 10:59:31.833870 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 10:59:31.835655 31878 raft_consensus.cc:1261] T 84d30203be3f44ba8ef0574d5847b979 P aa4643e48d204a2488ef3309fe7bc6d0 [term 1 FOLLOWER]: Refusing update from remote peer e744377d7a2c4927b9ce291768ea8c87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:59:31.835673 32140 raft_consensus.cc:1261] T 84d30203be3f44ba8ef0574d5847b979 P dfa54eee186e4771a6f1c52379ee9f8f [term 1 FOLLOWER]: Refusing update from remote peer e744377d7a2c4927b9ce291768ea8c87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:59:31.835990 32210 consensus_queue.cc:1035] T 84d30203be3f44ba8ef0574d5847b979 P e744377d7a2c4927b9ce291768ea8c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa4643e48d204a2488ef3309fe7bc6d0" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:59:31.836123 32207 consensus_queue.cc:1035] T 84d30203be3f44ba8ef0574d5847b979 P e744377d7a2c4927b9ce291768ea8c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfa54eee186e4771a6f1c52379ee9f8f" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:59:31.838701 32215 mvcc.cc:205] Tried to move back new op lower bound from 7334623526236147712 to 7334623526185463808. Current Snapshot: MvccSnapshot[applied={T|T < 7334623526236147712}]
I20260929 10:59:31.838790 32214 mvcc.cc:205] Tried to move back new op lower bound from 7334623526236147712 to 7334623526185463808. Current Snapshot: MvccSnapshot[applied={T|T < 7334623526236147712}]
I20260929 10:59:31.840602 31735 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:59:31.840708 31735 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:59:31.841451 31735 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e07daf8b0b8643fd8ada4d28ce5dccf5: Sending DeleteTablet for 3 replicas of tablet 84d30203be3f44ba8ef0574d5847b979
I20260929 10:59:31.841795 31858 tablet_service.cc:1514] Processing DeleteTablet for tablet 84d30203be3f44ba8ef0574d5847b979 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:31 UTC) from {username='slave'} at 127.0.0.1:42312
I20260929 10:59:31.841815 31989 tablet_service.cc:1514] Processing DeleteTablet for tablet 84d30203be3f44ba8ef0574d5847b979 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:31 UTC) from {username='slave'} at 127.0.0.1:48456
I20260929 10:59:31.841792 32120 tablet_service.cc:1514] Processing DeleteTablet for tablet 84d30203be3f44ba8ef0574d5847b979 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:31 UTC) from {username='slave'} at 127.0.0.1:59462
I20260929 10:59:31.842134 32222 tablet_replica.cc:330] T 84d30203be3f44ba8ef0574d5847b979 P aa4643e48d204a2488ef3309fe7bc6d0: stopping tablet replica
I20260929 10:59:31.842159 32223 tablet_replica.cc:330] T 84d30203be3f44ba8ef0574d5847b979 P dfa54eee186e4771a6f1c52379ee9f8f: stopping tablet replica
I20260929 10:59:31.842249 32222 raft_consensus.cc:2229] T 84d30203be3f44ba8ef0574d5847b979 P aa4643e48d204a2488ef3309fe7bc6d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:59:31.842314 32223 raft_consensus.cc:2229] T 84d30203be3f44ba8ef0574d5847b979 P dfa54eee186e4771a6f1c52379ee9f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:59:31.842347 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 31796
I20260929 10:59:31.842361 32222 raft_consensus.cc:2258] T 84d30203be3f44ba8ef0574d5847b979 P aa4643e48d204a2488ef3309fe7bc6d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:31.842450 32223 raft_consensus.cc:2258] T 84d30203be3f44ba8ef0574d5847b979 P dfa54eee186e4771a6f1c52379ee9f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:31.842927 32223 ts_tablet_manager.cc:1905] T 84d30203be3f44ba8ef0574d5847b979 P dfa54eee186e4771a6f1c52379ee9f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:31.844744 32224 tablet_replica.cc:330] T 84d30203be3f44ba8ef0574d5847b979 P e744377d7a2c4927b9ce291768ea8c87: stopping tablet replica
I20260929 10:59:31.844863 32224 raft_consensus.cc:2229] T 84d30203be3f44ba8ef0574d5847b979 P e744377d7a2c4927b9ce291768ea8c87 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:59:31.844971 32224 raft_consensus.cc:2258] T 84d30203be3f44ba8ef0574d5847b979 P e744377d7a2c4927b9ce291768ea8c87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:31.845021 32223 ts_tablet_manager.cc:1918] T 84d30203be3f44ba8ef0574d5847b979 P dfa54eee186e4771a6f1c52379ee9f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:59:31.845336 32223 log.cc:1196] T 84d30203be3f44ba8ef0574d5847b979 P dfa54eee186e4771a6f1c52379ee9f8f: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-2/wal/wals/84d30203be3f44ba8ef0574d5847b979
I20260929 10:59:31.845419 32224 ts_tablet_manager.cc:1905] T 84d30203be3f44ba8ef0574d5847b979 P e744377d7a2c4927b9ce291768ea8c87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:31.845619 32223 ts_tablet_manager.cc:1939] T 84d30203be3f44ba8ef0574d5847b979 P dfa54eee186e4771a6f1c52379ee9f8f: Deleting consensus metadata
I20260929 10:59:31.846112 31722 catalog_manager.cc:4841] TS dfa54eee186e4771a6f1c52379ee9f8f (127.26.120.195:45035): tablet 84d30203be3f44ba8ef0574d5847b979 (table test-table [id=636ad6849b884cdd87436e247deaba4a]) successfully deleted
I20260929 10:59:31.847193 32224 ts_tablet_manager.cc:1918] T 84d30203be3f44ba8ef0574d5847b979 P e744377d7a2c4927b9ce291768ea8c87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:59:31.847287 32224 log.cc:1196] T 84d30203be3f44ba8ef0574d5847b979 P e744377d7a2c4927b9ce291768ea8c87: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790679549986039-27107-0/minicluster-data/ts-1/wal/wals/84d30203be3f44ba8ef0574d5847b979
I20260929 10:59:31.847589 32224 ts_tablet_manager.cc:1939] T 84d30203be3f44ba8ef0574d5847b979 P e744377d7a2c4927b9ce291768ea8c87: Deleting consensus metadata
I20260929 10:59:31.847999 31722 catalog_manager.cc:4841] TS e744377d7a2c4927b9ce291768ea8c87 (127.26.120.194:34531): tablet 84d30203be3f44ba8ef0574d5847b979 (table test-table [id=636ad6849b884cdd87436e247deaba4a]) successfully deleted
W20260929 10:59:31.848875 31720 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.120.193:42647 (error 108)
I20260929 10:59:31.849364 32195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:31.849238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:42647 (local address 127.0.0.1:42328)
0929 10:59:31.849287 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:31.849328 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.193:42647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260929 10:59:31.849515 31720 catalog_manager.cc:4584] TS aa4643e48d204a2488ef3309fe7bc6d0 (127.26.120.193:42647): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 84d30203be3f44ba8ef0574d5847b979: Network error: Client connection negotiation failed: client connection to 127.26.120.193:42647: connect: Connection refused (error 111)
I20260929 10:59:31.850013 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 31927
I20260929 10:59:31.855808 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 32058
I20260929 10:59:31.860587 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 31705
2026-09-29T10:59:31Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (700 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260929 10:59:31.879868 27107 mini_oidc.cc:152] Starting JWKS server
I20260929 10:59:31.880937 27107 webserver.cc:460] Webserver started at https://127.0.0.1:36031/ using document root <none> and password file <none>
I20260929 10:59:31.881043 27107 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 10:59:31.881206 27107 webserver.cc:460] Webserver started at http://127.0.0.1:37889/ using document root <none> and password file <none>
2026-09-29T10:59:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:59:31Z Disabled control of system clock
I20260929 10:59:31.888751 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:34417
--webserver_interface=127.26.120.254
--webserver_port=0
--builtin_ntp_servers=127.26.120.212:37761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:34417
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36031/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790679549986039-27107-0/testchainca.cert with env {}
W20260929 10:59:31.966411 32231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:31.966605 32231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:31.966626 32231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:31.968108 32231 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:59:31.968168 32231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:31.968185 32231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:31.968262 32231 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:59:31.968279 32231 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 10:59:31.968312 32231 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 10:59:31.968350 32231 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36031/jwks.json
I20260929 10:59:31.969959 32231 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:37761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:34417
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:34417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36031/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790679549986039-27107-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:31.970242 32231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:31.970471 32231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:31.975194 32236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:31.975320 32231 server_base.cc:1034] running on GCE node
W20260929 10:59:31.975219 32239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:31.975350 32237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:31.975646 32231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:31.975940 32231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:31.977105 32231 hybrid_clock.cc:648] HybridClock initialized: now 1790679571977096 us; error 31 us; skew 500 ppm
I20260929 10:59:31.978349 32231 webserver.cc:460] Webserver started at http://127.26.120.254:44239/ using document root <none> and password file <none>
I20260929 10:59:31.978581 32231 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:31.978649 32231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:31.978796 32231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:31.979830 32231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "1013d647e0874a20a072c0a902c54a98"
format_stamp: "Formatted at 2026-09-29 10:59:31 on dist-test-slave-2lzq"
server_key: "39d4c917fdb5ecaad4029f2675204110"
server_key_iv: "614dd3549b6e9c7e33fcbca38ded81a8"
server_key_version: "encryptionkey@0"
I20260929 10:59:31.980201 32231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "1013d647e0874a20a072c0a902c54a98"
format_stamp: "Formatted at 2026-09-29 10:59:31 on dist-test-slave-2lzq"
server_key: "39d4c917fdb5ecaad4029f2675204110"
server_key_iv: "614dd3549b6e9c7e33fcbca38ded81a8"
server_key_version: "encryptionkey@0"
I20260929 10:59:31.981504 32231 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:59:31.982311 32245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:31.982493 32231 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:59:31.982589 32231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "1013d647e0874a20a072c0a902c54a98"
format_stamp: "Formatted at 2026-09-29 10:59:31 on dist-test-slave-2lzq"
server_key: "39d4c917fdb5ecaad4029f2675204110"
server_key_iv: "614dd3549b6e9c7e33fcbca38ded81a8"
server_key_version: "encryptionkey@0"
I20260929 10:59:31.982671 32231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:31.996567 32231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:31.996902 32231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:31.997061 32231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:32.001261 32231 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:34417
I20260929 10:59:32.001295 32297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:34417 every 8 connection(s)
I20260929 10:59:32.001644 32231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 10:59:32.002637 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 32231
I20260929 10:59:32.002650 32298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:32.002759 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 10:59:32.002938 27107 external_mini_cluster.cc:1428] Setting key 13fee33dd79fc680fe28b50c5f0a6b3a
I20260929 10:59:32.006085 32298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1013d647e0874a20a072c0a902c54a98: Bootstrap starting.
I20260929 10:59:32.006714 32298 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1013d647e0874a20a072c0a902c54a98: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:32.007019 32298 log.cc:824] T 00000000000000000000000000000000 P 1013d647e0874a20a072c0a902c54a98: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:32.007742 32298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1013d647e0874a20a072c0a902c54a98: No bootstrap required, opened a new log
I20260929 10:59:32.008921 32301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:32.003507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51356 (local address 127.26.120.254:34417)
0929 10:59:32.003716 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:32.003726 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:32.003760 (+ 34us) server_negotiation.cc:407] Connection header received
0929 10:59:32.004248 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:32.004258 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:32.004350 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:32.004515 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:32.004672 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:32.005981 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:32.006838 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:32.007391 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:32.007547 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:32.008220 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:32.008223 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:32.008224 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:59:32.008425 (+ 201us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:32.008467 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:59:32.008481 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:32.008565 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:32.008624 (+ 59us) server_negotiation.cc:299] Negotiation successful
0929 10:59:32.008701 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":59,"threads_started":1}
I20260929 10:59:32.009997 27107 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:59:32.010218 32298 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1013d647e0874a20a072c0a902c54a98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1013d647e0874a20a072c0a902c54a98" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34417 } }
I20260929 10:59:32.010375 32298 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1013d647e0874a20a072c0a902c54a98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:32.010397 32298 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1013d647e0874a20a072c0a902c54a98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1013d647e0874a20a072c0a902c54a98, State: Initialized, Role: FOLLOWER
I20260929 10:59:32.010535 32298 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1013d647e0874a20a072c0a902c54a98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1013d647e0874a20a072c0a902c54a98" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34417 } }
I20260929 10:59:32.010609 32298 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1013d647e0874a20a072c0a902c54a98 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:59:32.010632 32298 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1013d647e0874a20a072c0a902c54a98 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:59:32.010699 32298 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1013d647e0874a20a072c0a902c54a98 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:32.011303 32298 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1013d647e0874a20a072c0a902c54a98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1013d647e0874a20a072c0a902c54a98" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34417 } }
I20260929 10:59:32.011426 32298 leader_election.cc:302] T 00000000000000000000000000000000 P 1013d647e0874a20a072c0a902c54a98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1013d647e0874a20a072c0a902c54a98; no voters:
I20260929 10:59:32.011579 32298 leader_election.cc:288] T 00000000000000000000000000000000 P 1013d647e0874a20a072c0a902c54a98 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:59:32.011624 32303 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1013d647e0874a20a072c0a902c54a98 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:59:32.011775 32303 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1013d647e0874a20a072c0a902c54a98 [term 1 LEADER]: Becoming Leader. State: Replica: 1013d647e0874a20a072c0a902c54a98, State: Running, Role: LEADER
I20260929 10:59:32.011852 32298 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1013d647e0874a20a072c0a902c54a98 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:59:32.011857 32303 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1013d647e0874a20a072c0a902c54a98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1013d647e0874a20a072c0a902c54a98" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34417 } }
I20260929 10:59:32.012233 32304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1013d647e0874a20a072c0a902c54a98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1013d647e0874a20a072c0a902c54a98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1013d647e0874a20a072c0a902c54a98" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34417 } } }
I20260929 10:59:32.012243 32305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1013d647e0874a20a072c0a902c54a98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1013d647e0874a20a072c0a902c54a98. Latest consensus state: current_term: 1 leader_uuid: "1013d647e0874a20a072c0a902c54a98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1013d647e0874a20a072c0a902c54a98" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34417 } } }
I20260929 10:59:32.012367 32305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1013d647e0874a20a072c0a902c54a98 [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:32.012553 32304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1013d647e0874a20a072c0a902c54a98 [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:32.012871 32310 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:59:32.013376 32310 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:59:32.014930 32310 catalog_manager.cc:1347] Generated new cluster ID: 77439b88dcba44589bf67cb65799fe4b
I20260929 10:59:32.014997 32310 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:59:32.023648 32310 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:59:32.024335 32310 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:59:32.035126 32310 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1013d647e0874a20a072c0a902c54a98: Generated new TSK 0
I20260929 10:59:32.035310 32310 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260929 10:59:32.057798 32328 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260929 10:59:32.057910 32328 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260929 10:59:32.058159 32301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:32.042637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51358 (local address 127.26.120.254:34417)
0929 10:59:32.042704 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:32.042706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:32.042860 (+ 154us) server_negotiation.cc:407] Connection header received
0929 10:59:32.042926 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:32.042927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:32.042941 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:32.043030 (+ 89us) server_negotiation.cc:226] Negotiated authn=JWT
0929 10:59:32.043166 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:32.043956 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:32.044696 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:32.045030 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:32.045100 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:32.045285 (+ 185us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 10:59:32.057882 (+ 12597us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36031/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0929 10:59:32.058047 (+ 165us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51358: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36031/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":42}
W20260929 10:59:32.058238 32301 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51358: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36031/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260929 10:59:32.058656 32327 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:32.042571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34417 (local address 127.0.0.1:51358)
0929 10:59:32.042789 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:32.042798 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:32.042856 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:32.043041 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:32.043042 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:32.043044 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0929 10:59:32.043102 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:32.043104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:32.044030 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:32.044030 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:32.044616 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:32.044617 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:32.045118 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:32.045119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:32.045168 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:32.045184 (+ 16us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 10:59:32.058129 (+ 12945us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36031/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0929 10:59:32.058263 (+ 134us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.26.120.254:34417: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36031/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":102,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.26.120.254:34417: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36031/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260929 10:59:32.059303 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 32231
2026-09-29T10:59:32Z chronyd exiting
I20260929 10:59:32.095423 27107 test_util.cc:175] -----------------------------------------------
I20260929 10:59:32.095507 27107 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790679549986039-27107-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (216 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260929 10:59:32.096282 27107 mini_oidc.cc:152] Starting JWKS server
I20260929 10:59:32.097002 27107 webserver.cc:460] Webserver started at https://127.0.0.1:39263/ using document root <none> and password file <none>
I20260929 10:59:32.097044 27107 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 10:59:32.097239 27107 webserver.cc:460] Webserver started at http://127.0.0.1:40499/ using document root <none> and password file <none>
2026-09-29T10:59:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:59:32Z Disabled control of system clock
I20260929 10:59:32.105028 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:42911
--webserver_interface=127.26.120.254
--webserver_port=0
--builtin_ntp_servers=127.26.120.212:34963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:42911
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39263/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790679549986039-27107-0/testchainca.cert with env {}
W20260929 10:59:32.185577 32334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:32.185773 32334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:32.185796 32334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:32.187064 32334 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:59:32.187107 32334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:32.187120 32334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:32.187134 32334 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:59:32.187186 32334 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 10:59:32.187203 32334 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 10:59:32.187227 32334 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39263/jwks.json
I20260929 10:59:32.188730 32334 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:34963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:42911
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:42911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39263/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790679549986039-27107-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:32.188992 32334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:32.189217 32334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:32.194078 32339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:32.194114 32340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:32.194154 32334 server_base.cc:1034] running on GCE node
W20260929 10:59:32.194119 32342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:32.194489 32334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:32.194833 32334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:32.196008 32334 hybrid_clock.cc:648] HybridClock initialized: now 1790679572196002 us; error 36 us; skew 500 ppm
I20260929 10:59:32.197177 32334 webserver.cc:460] Webserver started at http://127.26.120.254:43747/ using document root <none> and password file <none>
I20260929 10:59:32.197440 32334 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:32.197513 32334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:32.197675 32334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:32.198841 32334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "6e8baa7c11974520a2a428cd042a892b"
format_stamp: "Formatted at 2026-09-29 10:59:32 on dist-test-slave-2lzq"
server_key: "546e1474c0f7159e8a65570b8b546835"
server_key_iv: "1b801d020c4f01d4dbdb85fa08a86277"
server_key_version: "encryptionkey@0"
I20260929 10:59:32.199208 32334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "6e8baa7c11974520a2a428cd042a892b"
format_stamp: "Formatted at 2026-09-29 10:59:32 on dist-test-slave-2lzq"
server_key: "546e1474c0f7159e8a65570b8b546835"
server_key_iv: "1b801d020c4f01d4dbdb85fa08a86277"
server_key_version: "encryptionkey@0"
I20260929 10:59:32.200508 32334 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 10:59:32.201264 32348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:32.201431 32334 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:59:32.201531 32334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "6e8baa7c11974520a2a428cd042a892b"
format_stamp: "Formatted at 2026-09-29 10:59:32 on dist-test-slave-2lzq"
server_key: "546e1474c0f7159e8a65570b8b546835"
server_key_iv: "1b801d020c4f01d4dbdb85fa08a86277"
server_key_version: "encryptionkey@0"
I20260929 10:59:32.201618 32334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:32.235515 32334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:32.235862 32334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:32.236022 32334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:32.240100 32334 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:42911
I20260929 10:59:32.240134 32400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:42911 every 8 connection(s)
I20260929 10:59:32.240491 32334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 10:59:32.241168 32401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:32.243860 32401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e8baa7c11974520a2a428cd042a892b: Bootstrap starting.
I20260929 10:59:32.244376 32401 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e8baa7c11974520a2a428cd042a892b: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:32.244601 32401 log.cc:824] T 00000000000000000000000000000000 P 6e8baa7c11974520a2a428cd042a892b: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:32.245232 32401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e8baa7c11974520a2a428cd042a892b: No bootstrap required, opened a new log
I20260929 10:59:32.246802 32401 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e8baa7c11974520a2a428cd042a892b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e8baa7c11974520a2a428cd042a892b" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 42911 } }
I20260929 10:59:32.246932 32401 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e8baa7c11974520a2a428cd042a892b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:32.246954 32401 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e8baa7c11974520a2a428cd042a892b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e8baa7c11974520a2a428cd042a892b, State: Initialized, Role: FOLLOWER
I20260929 10:59:32.247059 32401 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e8baa7c11974520a2a428cd042a892b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e8baa7c11974520a2a428cd042a892b" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 42911 } }
I20260929 10:59:32.247107 32401 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e8baa7c11974520a2a428cd042a892b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:59:32.247126 32401 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e8baa7c11974520a2a428cd042a892b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:59:32.247155 32401 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e8baa7c11974520a2a428cd042a892b [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:32.247609 32401 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e8baa7c11974520a2a428cd042a892b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e8baa7c11974520a2a428cd042a892b" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 42911 } }
I20260929 10:59:32.247706 32401 leader_election.cc:302] T 00000000000000000000000000000000 P 6e8baa7c11974520a2a428cd042a892b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e8baa7c11974520a2a428cd042a892b; no voters:
I20260929 10:59:32.247822 32401 leader_election.cc:288] T 00000000000000000000000000000000 P 6e8baa7c11974520a2a428cd042a892b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:59:32.247861 32404 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e8baa7c11974520a2a428cd042a892b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:59:32.248029 32404 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e8baa7c11974520a2a428cd042a892b [term 1 LEADER]: Becoming Leader. State: Replica: 6e8baa7c11974520a2a428cd042a892b, State: Running, Role: LEADER
I20260929 10:59:32.248090 32401 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e8baa7c11974520a2a428cd042a892b [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:59:32.248123 32404 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e8baa7c11974520a2a428cd042a892b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e8baa7c11974520a2a428cd042a892b" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 42911 } }
I20260929 10:59:32.248520 32405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e8baa7c11974520a2a428cd042a892b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e8baa7c11974520a2a428cd042a892b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e8baa7c11974520a2a428cd042a892b" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 42911 } } }
I20260929 10:59:32.248670 32405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e8baa7c11974520a2a428cd042a892b [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:32.248704 32406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e8baa7c11974520a2a428cd042a892b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e8baa7c11974520a2a428cd042a892b. Latest consensus state: current_term: 1 leader_uuid: "6e8baa7c11974520a2a428cd042a892b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e8baa7c11974520a2a428cd042a892b" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 42911 } } }
I20260929 10:59:32.248764 32406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e8baa7c11974520a2a428cd042a892b [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:32.249269 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 32334
I20260929 10:59:32.249369 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 10:59:32.249570 27107 external_mini_cluster.cc:1428] Setting key 7e443e5eeadd3fb4a04f7d21a17e421f
W20260929 10:59:32.250679 32420 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6e8baa7c11974520a2a428cd042a892b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:59:32.250721 32420 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:59:32.250792 32413 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:59:32.250988 32413 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:59:32.252501 32413 catalog_manager.cc:1347] Generated new cluster ID: 8e7f19b10bee401587cb52e1f80c328b
I20260929 10:59:32.252552 32413 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:59:32.256098 32422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:32.250193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41264 (local address 127.26.120.254:42911)
0929 10:59:32.250468 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:32.250477 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:32.250870 (+ 393us) server_negotiation.cc:407] Connection header received
0929 10:59:32.251106 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:32.251114 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:32.251207 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:32.251318 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:32.251661 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:32.252826 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:32.253933 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:32.254529 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:32.254636 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:32.255342 (+ 706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:32.255345 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:32.255345 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:59:32.255543 (+ 198us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:32.255583 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:59:32.255596 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:32.255656 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:32.255752 (+ 96us) server_negotiation.cc:299] Negotiation successful
0929 10:59:32.255902 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":69,"threads_started":1}
I20260929 10:59:32.256950 27107 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:59:32.273595 32413 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:59:32.274315 32413 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:59:32.287768 32413 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e8baa7c11974520a2a428cd042a892b: Generated new TSK 0
I20260929 10:59:32.287994 32413 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260929 10:59:32.317803 32431 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260929 10:59:32.317909 32431 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260929 10:59:32.318228 32422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:32.296371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41274 (local address 127.26.120.254:42911)
0929 10:59:32.296443 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:32.296445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:32.296451 (+ 6us) server_negotiation.cc:407] Connection header received
0929 10:59:32.296479 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:32.296480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:32.296490 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:32.296594 (+ 104us) server_negotiation.cc:226] Negotiated authn=JWT
0929 10:59:32.296707 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:32.297206 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:32.298470 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:32.298669 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:32.298722 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:32.298905 (+ 183us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 10:59:32.317958 (+ 19053us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39263/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0929 10:59:32.318155 (+ 197us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41274: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39263/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":45}
W20260929 10:59:32.318296 32430 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:32.296152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:42911 (local address 127.0.0.1:41274)
0929 10:59:32.296290 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:32.296300 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:32.296354 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:32.296573 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:32.296574 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:32.296577 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0929 10:59:32.296647 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:32.296650 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:32.297348 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:32.297349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:32.298390 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:32.298393 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:32.298748 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:32.298749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:32.298811 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:32.298816 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 10:59:32.318089 (+ 19273us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39263/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0929 10:59:32.318200 (+ 111us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.26.120.254:42911: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39263/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":93,"threads_started":1}
W20260929 10:59:32.318303 32422 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41274: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39263/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260929 10:59:32.318672 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 32334
2026-09-29T10:59:32Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (259 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260929 10:59:32.355661 27107 mini_oidc.cc:152] Starting JWKS server
I20260929 10:59:32.356315 27107 webserver.cc:460] Webserver started at https://127.0.0.1:44869/ using document root <none> and password file <none>
I20260929 10:59:32.356353 27107 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 10:59:32.356491 27107 webserver.cc:460] Webserver started at http://127.0.0.1:40185/ using document root <none> and password file <none>
2026-09-29T10:59:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:59:32Z Disabled control of system clock
I20260929 10:59:32.364141 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:34325
--webserver_interface=127.26.120.254
--webserver_port=0
--builtin_ntp_servers=127.26.120.212:33865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:34325
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44869/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260929 10:59:32.442996 32437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:32.443250 32437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:32.443285 32437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:32.444434 32437 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:59:32.444511 32437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:32.444531 32437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:32.444552 32437 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:59:32.444581 32437 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 10:59:32.444612 32437 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 10:59:32.444629 32437 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44869/jwks.json
I20260929 10:59:32.445976 32437 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:33865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:34325
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:34325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44869/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:32.446278 32437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:32.446569 32437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:32.451340 32443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:32.451350 32442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:32.451529 32437 server_base.cc:1034] running on GCE node
W20260929 10:59:32.451356 32445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:32.451851 32437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:32.452157 32437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:32.453332 32437 hybrid_clock.cc:648] HybridClock initialized: now 1790679572453328 us; error 36 us; skew 500 ppm
I20260929 10:59:32.461900 32437 webserver.cc:460] Webserver started at http://127.26.120.254:35051/ using document root <none> and password file <none>
I20260929 10:59:32.462220 32437 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:32.462294 32437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:32.462445 32437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:32.463515 32437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "fc532ce6f95747359bcc8d0a48f38829"
format_stamp: "Formatted at 2026-09-29 10:59:32 on dist-test-slave-2lzq"
server_key: "940d75865a896476dd7b96e188fedd06"
server_key_iv: "0fe0e7843773785c256f1d5cb06b4227"
server_key_version: "encryptionkey@0"
I20260929 10:59:32.463878 32437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "fc532ce6f95747359bcc8d0a48f38829"
format_stamp: "Formatted at 2026-09-29 10:59:32 on dist-test-slave-2lzq"
server_key: "940d75865a896476dd7b96e188fedd06"
server_key_iv: "0fe0e7843773785c256f1d5cb06b4227"
server_key_version: "encryptionkey@0"
I20260929 10:59:32.465163 32437 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:59:32.465890 32451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:32.466121 32437 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:59:32.466212 32437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "fc532ce6f95747359bcc8d0a48f38829"
format_stamp: "Formatted at 2026-09-29 10:59:32 on dist-test-slave-2lzq"
server_key: "940d75865a896476dd7b96e188fedd06"
server_key_iv: "0fe0e7843773785c256f1d5cb06b4227"
server_key_version: "encryptionkey@0"
I20260929 10:59:32.466266 32437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:32.482857 32437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:32.483114 32437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:32.483213 32437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:32.486910 32437 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:34325
I20260929 10:59:32.486953 32503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:34325 every 8 connection(s)
I20260929 10:59:32.487317 32437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 10:59:32.487977 32504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:32.488240 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 32437
I20260929 10:59:32.488349 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 10:59:32.488524 27107 external_mini_cluster.cc:1428] Setting key be275fac70a34e5cf751bccba2d4f72c
I20260929 10:59:32.491652 32504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc532ce6f95747359bcc8d0a48f38829: Bootstrap starting.
I20260929 10:59:32.492290 32504 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc532ce6f95747359bcc8d0a48f38829: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:32.492626 32504 log.cc:824] T 00000000000000000000000000000000 P fc532ce6f95747359bcc8d0a48f38829: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:32.493422 32504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc532ce6f95747359bcc8d0a48f38829: No bootstrap required, opened a new log
I20260929 10:59:32.493978 32507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:32.489055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34120 (local address 127.26.120.254:34325)
0929 10:59:32.489237 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:32.489244 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:32.489268 (+ 24us) server_negotiation.cc:407] Connection header received
0929 10:59:32.489656 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:32.489663 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:32.489722 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:32.489834 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:32.490137 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:32.490985 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:32.492048 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:32.492487 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:32.492580 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:32.493311 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:32.493315 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:32.493315 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:59:32.493500 (+ 185us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:32.493556 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:59:32.493568 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:32.493617 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:32.493698 (+ 81us) server_negotiation.cc:299] Negotiation successful
0929 10:59:32.493780 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":78,"threads_started":1}
I20260929 10:59:32.494916 27107 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:59:32.495586 32504 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc532ce6f95747359bcc8d0a48f38829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc532ce6f95747359bcc8d0a48f38829" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34325 } }
I20260929 10:59:32.495764 32504 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc532ce6f95747359bcc8d0a48f38829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:32.495815 32504 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc532ce6f95747359bcc8d0a48f38829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc532ce6f95747359bcc8d0a48f38829, State: Initialized, Role: FOLLOWER
I20260929 10:59:32.495932 32504 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc532ce6f95747359bcc8d0a48f38829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc532ce6f95747359bcc8d0a48f38829" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34325 } }
I20260929 10:59:32.496007 32504 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc532ce6f95747359bcc8d0a48f38829 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:59:32.496029 32504 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc532ce6f95747359bcc8d0a48f38829 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:59:32.496057 32504 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc532ce6f95747359bcc8d0a48f38829 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:32.496531 32504 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc532ce6f95747359bcc8d0a48f38829 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc532ce6f95747359bcc8d0a48f38829" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34325 } }
I20260929 10:59:32.496610 32504 leader_election.cc:302] T 00000000000000000000000000000000 P fc532ce6f95747359bcc8d0a48f38829 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc532ce6f95747359bcc8d0a48f38829; no voters:
I20260929 10:59:32.496728 32504 leader_election.cc:288] T 00000000000000000000000000000000 P fc532ce6f95747359bcc8d0a48f38829 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:59:32.496802 32509 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc532ce6f95747359bcc8d0a48f38829 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:59:32.496938 32509 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc532ce6f95747359bcc8d0a48f38829 [term 1 LEADER]: Becoming Leader. State: Replica: fc532ce6f95747359bcc8d0a48f38829, State: Running, Role: LEADER
I20260929 10:59:32.497013 32509 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc532ce6f95747359bcc8d0a48f38829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc532ce6f95747359bcc8d0a48f38829" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34325 } }
I20260929 10:59:32.497159 32504 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc532ce6f95747359bcc8d0a48f38829 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:59:32.497326 32510 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc532ce6f95747359bcc8d0a48f38829 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc532ce6f95747359bcc8d0a48f38829" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc532ce6f95747359bcc8d0a48f38829" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34325 } } }
I20260929 10:59:32.497346 32511 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc532ce6f95747359bcc8d0a48f38829 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc532ce6f95747359bcc8d0a48f38829. Latest consensus state: current_term: 1 leader_uuid: "fc532ce6f95747359bcc8d0a48f38829" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc532ce6f95747359bcc8d0a48f38829" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34325 } } }
I20260929 10:59:32.497438 32510 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc532ce6f95747359bcc8d0a48f38829 [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:32.497447 32511 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc532ce6f95747359bcc8d0a48f38829 [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:32.497648 32513 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:59:32.498178 32513 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:59:32.499800 32513 catalog_manager.cc:1347] Generated new cluster ID: 2d44bbebcff14adeb49378ca813e2be3
I20260929 10:59:32.499843 32513 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:59:32.507246 32513 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:59:32.507738 32513 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:59:32.515480 32513 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc532ce6f95747359bcc8d0a48f38829: Generated new TSK 0
I20260929 10:59:32.515671 32513 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:59:32.526917 32507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:32.523674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34124 (local address 127.26.120.254:34325)
0929 10:59:32.523720 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:32.523722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:32.523798 (+ 76us) server_negotiation.cc:407] Connection header received
0929 10:59:32.523899 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:32.523900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:32.523912 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:32.523977 (+ 65us) server_negotiation.cc:226] Negotiated authn=JWT
0929 10:59:32.524133 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:32.524633 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:32.525420 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:32.525634 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:32.525719 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:32.525895 (+ 176us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 10:59:32.526692 (+ 797us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44869/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0929 10:59:32.526839 (+ 147us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34124: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44869/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":26}
W20260929 10:59:32.526979 32507 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34124: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44869/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260929 10:59:32.527096 32533 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:32.523585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34325 (local address 127.0.0.1:34124)
0929 10:59:32.523719 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:32.523730 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:32.523816 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:32.523987 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:32.523987 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:32.523989 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0929 10:59:32.524060 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:32.524062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:32.524716 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:32.524717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:32.525315 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:32.525317 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:32.525727 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:32.525728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:32.525791 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:32.525795 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 10:59:32.526790 (+ 995us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44869/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0929 10:59:32.526876 (+ 86us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.26.120.254:34325: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44869/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":85,"threads_started":1}
I20260929 10:59:32.527367 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 32437
E20260929 10:59:32.538012 32534 webserver.cc:533] Webserver: sslize failed:
E20260929 10:59:32.538120 32534 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-29T10:59:32Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (208 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-29T10:59:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:59:32Z Disabled control of system clock
I20260929 10:59:32.572360 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:35507
--webserver_interface=127.26.120.254
--webserver_port=0
--builtin_ntp_servers=127.26.120.212:33177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:35507
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790679549986039-27107-0/testchainca.cert with env {}
W20260929 10:59:32.650936 32538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:32.651144 32538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:32.651166 32538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:32.652272 32538 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:59:32.652309 32538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:32.652323 32538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:32.652335 32538 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:59:32.652347 32538 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 10:59:32.652359 32538 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 10:59:32.652370 32538 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260929 10:59:32.653801 32538 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:33177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:35507
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:35507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790679549986039-27107-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:32.654002 32538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:32.654256 32538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:32.659320 32544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:32.659324 32546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:32.659358 32538 server_base.cc:1034] running on GCE node
W20260929 10:59:32.659324 32543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:32.659795 32538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:32.660108 32538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:32.661275 32538 hybrid_clock.cc:648] HybridClock initialized: now 1790679572661262 us; error 25 us; skew 500 ppm
I20260929 10:59:32.662690 32538 webserver.cc:460] Webserver started at http://127.26.120.254:36517/ using document root <none> and password file <none>
I20260929 10:59:32.662938 32538 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:32.663012 32538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:32.663164 32538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:32.664237 32538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "4e5b3a81050140c3a1787020ee7ed1e1"
format_stamp: "Formatted at 2026-09-29 10:59:32 on dist-test-slave-2lzq"
server_key: "b39a4fcdb20d30b36cc4371244097619"
server_key_iv: "af2b1e1d2d14a850acd1b3125576f276"
server_key_version: "encryptionkey@0"
I20260929 10:59:32.664624 32538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "4e5b3a81050140c3a1787020ee7ed1e1"
format_stamp: "Formatted at 2026-09-29 10:59:32 on dist-test-slave-2lzq"
server_key: "b39a4fcdb20d30b36cc4371244097619"
server_key_iv: "af2b1e1d2d14a850acd1b3125576f276"
server_key_version: "encryptionkey@0"
I20260929 10:59:32.665858 32538 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 10:59:32.666652 32552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:32.666819 32538 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:32.666916 32538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "4e5b3a81050140c3a1787020ee7ed1e1"
format_stamp: "Formatted at 2026-09-29 10:59:32 on dist-test-slave-2lzq"
server_key: "b39a4fcdb20d30b36cc4371244097619"
server_key_iv: "af2b1e1d2d14a850acd1b3125576f276"
server_key_version: "encryptionkey@0"
I20260929 10:59:32.666996 32538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:32.678226 32538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:32.678514 32538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:32.678675 32538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:32.682597 32538 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:35507
I20260929 10:59:32.682644 32604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:35507 every 8 connection(s)
I20260929 10:59:32.682978 32538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 10:59:32.683887 32605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:32.685985 32605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e5b3a81050140c3a1787020ee7ed1e1: Bootstrap starting.
I20260929 10:59:32.686534 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 32538
I20260929 10:59:32.686638 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 10:59:32.686724 32605 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e5b3a81050140c3a1787020ee7ed1e1: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:32.686834 27107 external_mini_cluster.cc:1428] Setting key 99b065e798271a9946ee1d386e235c33
I20260929 10:59:32.687063 32605 log.cc:824] T 00000000000000000000000000000000 P 4e5b3a81050140c3a1787020ee7ed1e1: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:32.687762 32605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e5b3a81050140c3a1787020ee7ed1e1: No bootstrap required, opened a new log
I20260929 10:59:32.689955 32605 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e5b3a81050140c3a1787020ee7ed1e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e5b3a81050140c3a1787020ee7ed1e1" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 35507 } }
I20260929 10:59:32.690148 32605 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e5b3a81050140c3a1787020ee7ed1e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:32.690182 32605 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e5b3a81050140c3a1787020ee7ed1e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e5b3a81050140c3a1787020ee7ed1e1, State: Initialized, Role: FOLLOWER
I20260929 10:59:32.690297 32605 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e5b3a81050140c3a1787020ee7ed1e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e5b3a81050140c3a1787020ee7ed1e1" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 35507 } }
I20260929 10:59:32.690378 32605 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e5b3a81050140c3a1787020ee7ed1e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:59:32.690409 32605 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e5b3a81050140c3a1787020ee7ed1e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:59:32.690470 32605 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e5b3a81050140c3a1787020ee7ed1e1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:32.691092 32605 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e5b3a81050140c3a1787020ee7ed1e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e5b3a81050140c3a1787020ee7ed1e1" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 35507 } }
I20260929 10:59:32.691205 32605 leader_election.cc:302] T 00000000000000000000000000000000 P 4e5b3a81050140c3a1787020ee7ed1e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e5b3a81050140c3a1787020ee7ed1e1; no voters:
I20260929 10:59:32.691375 32605 leader_election.cc:288] T 00000000000000000000000000000000 P 4e5b3a81050140c3a1787020ee7ed1e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:59:32.691427 32610 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e5b3a81050140c3a1787020ee7ed1e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:59:32.691581 32610 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e5b3a81050140c3a1787020ee7ed1e1 [term 1 LEADER]: Becoming Leader. State: Replica: 4e5b3a81050140c3a1787020ee7ed1e1, State: Running, Role: LEADER
I20260929 10:59:32.691668 32605 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e5b3a81050140c3a1787020ee7ed1e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:59:32.691691 32610 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e5b3a81050140c3a1787020ee7ed1e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e5b3a81050140c3a1787020ee7ed1e1" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 35507 } }
I20260929 10:59:32.692016 32610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e5b3a81050140c3a1787020ee7ed1e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e5b3a81050140c3a1787020ee7ed1e1. Latest consensus state: current_term: 1 leader_uuid: "4e5b3a81050140c3a1787020ee7ed1e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e5b3a81050140c3a1787020ee7ed1e1" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 35507 } } }
I20260929 10:59:32.692096 32610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e5b3a81050140c3a1787020ee7ed1e1 [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:32.692479 32611 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e5b3a81050140c3a1787020ee7ed1e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e5b3a81050140c3a1787020ee7ed1e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e5b3a81050140c3a1787020ee7ed1e1" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 35507 } } }
I20260929 10:59:32.692559 32611 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e5b3a81050140c3a1787020ee7ed1e1 [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:32.692670 32616 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:59:32.693028 32609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:32.687363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33008 (local address 127.26.120.254:35507)
0929 10:59:32.687584 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:32.687598 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:32.687623 (+ 25us) server_negotiation.cc:407] Connection header received
0929 10:59:32.688109 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:32.688116 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:32.688175 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:32.688291 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:32.688508 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:32.689360 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:32.690483 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:32.690967 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:32.691069 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:32.691976 (+ 907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:32.691980 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:32.691980 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 10:59:32.692260 (+ 280us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:32.692335 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 10:59:32.692355 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:32.692440 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:32.692615 (+ 175us) server_negotiation.cc:299] Negotiation successful
0929 10:59:32.692758 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":102,"threads_started":1}
I20260929 10:59:32.693190 32616 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:59:32.694635 27107 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 10:59:32.695904 32616 catalog_manager.cc:1347] Generated new cluster ID: ec43b0a8322c4d4483bd004e0b3da6a5
I20260929 10:59:32.695973 32616 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:59:32.716975 32616 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:59:32.717561 32616 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:59:32.722046 32616 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e5b3a81050140c3a1787020ee7ed1e1: Generated new TSK 0
I20260929 10:59:32.722270 32616 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:59:32.760475 32609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:32.733621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33010 (local address 127.26.120.254:35507)
0929 10:59:32.733668 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:32.733671 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:32.733756 (+ 85us) server_negotiation.cc:407] Connection header received
0929 10:59:32.733800 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:32.733801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:32.733813 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:32.733895 (+ 82us) server_negotiation.cc:226] Negotiated authn=JWT
0929 10:59:32.734010 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:32.734839 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:32.735585 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:32.735905 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:32.735975 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:32.736120 (+ 145us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 10:59:32.760206 (+ 24086us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0929 10:59:32.760398 (+ 192us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33010: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":25}
W20260929 10:59:32.760562 32609 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33010: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260929 10:59:32.760593 32634 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:32.733536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:35507 (local address 127.0.0.1:33010)
0929 10:59:32.733681 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:32.733691 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:32.733749 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:32.733898 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:32.733899 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:32.733900 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0929 10:59:32.733950 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:32.733952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:32.734913 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:32.734914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:32.735494 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:32.735496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:32.735982 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:32.735983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:32.736025 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:32.736029 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 10:59:32.760361 (+ 24332us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0929 10:59:32.760479 (+ 118us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.26.120.254:35507: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":96,"threads_started":1}
I20260929 10:59:32.761029 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 32538
2026-09-29T10:59:32Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (213 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (87 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (87 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:59:32 dist-test-slave-2lzq krb5kdc[32646](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:59:32 dist-test-slave-2lzq krb5kdc[32646](info): set up 2 sockets
Sep 29 10:59:32 dist-test-slave-2lzq krb5kdc[32646](info): commencing operation
krb5kdc: starting...
W20260929 10:59:33.966446 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 10:59:34 dist-test-slave-2lzq krb5kdc[32646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:59:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:59:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.120.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
I20260929 10:59:34.083225 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:37853
--webserver_interface=127.26.120.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:43963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:37853
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:34.163864 32659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:34.164064 32659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:34.164086 32659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:34.165181 32659 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:59:34.165217 32659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:34.165231 32659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:34.165244 32659 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:59:34.165256 32659 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:59:34.166656 32659 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:43963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:37853
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:37853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:34.166846 32659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:34.167006 32659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:34.171989 32667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:34.172025 32664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:34.172016 32659 server_base.cc:1034] running on GCE node
W20260929 10:59:34.171989 32665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:34.172539 32659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:34.172832 32659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:34.173996 32659 hybrid_clock.cc:648] HybridClock initialized: now 1790679574173983 us; error 30 us; skew 500 ppm
Sep 29 10:59:34 dist-test-slave-2lzq krb5kdc[32646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679574, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:34.176497 32659 init.cc:377] Logged in from keytab as kudu/127.26.120.254@KRBTEST.COM (short username kudu)
I20260929 10:59:34.177098 32659 webserver.cc:460] Webserver started at http://127.26.120.254:37309/ using document root <none> and password file <none>
I20260929 10:59:34.177454 32659 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:34.177524 32659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:34.177673 32659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:34.179137 32659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "8a96ad39e7ee4234b562954bf72642d5"
format_stamp: "Formatted at 2026-09-29 10:59:34 on dist-test-slave-2lzq"
server_key: "8d9716452bb87c285012ff30550f1d77"
server_key_iv: "a1bc7a8002626fb37be1ace44f066d43"
server_key_version: "encryptionkey@0"
I20260929 10:59:34.179567 32659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "8a96ad39e7ee4234b562954bf72642d5"
format_stamp: "Formatted at 2026-09-29 10:59:34 on dist-test-slave-2lzq"
server_key: "8d9716452bb87c285012ff30550f1d77"
server_key_iv: "a1bc7a8002626fb37be1ace44f066d43"
server_key_version: "encryptionkey@0"
I20260929 10:59:34.181058 32659 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:59:34.181726 32674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:34.181928 32659 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:59:34.182016 32659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "8a96ad39e7ee4234b562954bf72642d5"
format_stamp: "Formatted at 2026-09-29 10:59:34 on dist-test-slave-2lzq"
server_key: "8d9716452bb87c285012ff30550f1d77"
server_key_iv: "a1bc7a8002626fb37be1ace44f066d43"
server_key_version: "encryptionkey@0"
I20260929 10:59:34.182117 32659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:34.189946 32659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:34.192222 32659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:34.192337 32659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:34.196193 32659 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:37853
I20260929 10:59:34.196257 32726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:37853 every 8 connection(s)
I20260929 10:59:34.196607 32659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 10:59:34.197510 32727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:34.197953 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 32659
I20260929 10:59:34.198098 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 10:59:34.198283 27107 external_mini_cluster.cc:1428] Setting key a7bd3c6f019256027a38d51a7f25375d
I20260929 10:59:34.200697 32727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a96ad39e7ee4234b562954bf72642d5: Bootstrap starting.
I20260929 10:59:34.201373 32727 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a96ad39e7ee4234b562954bf72642d5: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:34.201669 32727 log.cc:824] T 00000000000000000000000000000000 P 8a96ad39e7ee4234b562954bf72642d5: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:34.202456 32727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a96ad39e7ee4234b562954bf72642d5: No bootstrap required, opened a new log
I20260929 10:59:34.204636 32727 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a96ad39e7ee4234b562954bf72642d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a96ad39e7ee4234b562954bf72642d5" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 37853 } }
I20260929 10:59:34.204774 32727 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a96ad39e7ee4234b562954bf72642d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:34.204797 32727 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a96ad39e7ee4234b562954bf72642d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a96ad39e7ee4234b562954bf72642d5, State: Initialized, Role: FOLLOWER
I20260929 10:59:34.204960 32727 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a96ad39e7ee4234b562954bf72642d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a96ad39e7ee4234b562954bf72642d5" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 37853 } }
I20260929 10:59:34.205058 32727 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a96ad39e7ee4234b562954bf72642d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:59:34.205089 32727 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a96ad39e7ee4234b562954bf72642d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:59:34.205134 32727 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a96ad39e7ee4234b562954bf72642d5 [term 0 FOLLOWER]: Advancing to term 1
Sep 29 10:59:34 dist-test-slave-2lzq krb5kdc[32646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:34.205842 32727 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a96ad39e7ee4234b562954bf72642d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a96ad39e7ee4234b562954bf72642d5" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 37853 } }
I20260929 10:59:34.205971 32727 leader_election.cc:302] T 00000000000000000000000000000000 P 8a96ad39e7ee4234b562954bf72642d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a96ad39e7ee4234b562954bf72642d5; no voters:
I20260929 10:59:34.206171 32727 leader_election.cc:288] T 00000000000000000000000000000000 P 8a96ad39e7ee4234b562954bf72642d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:59:34.206220 32732 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a96ad39e7ee4234b562954bf72642d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:59:34.206343 32732 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a96ad39e7ee4234b562954bf72642d5 [term 1 LEADER]: Becoming Leader. State: Replica: 8a96ad39e7ee4234b562954bf72642d5, State: Running, Role: LEADER
I20260929 10:59:34.206454 32727 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a96ad39e7ee4234b562954bf72642d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:59:34.206432 32732 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a96ad39e7ee4234b562954bf72642d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a96ad39e7ee4234b562954bf72642d5" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 37853 } }
I20260929 10:59:34.206835 32732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a96ad39e7ee4234b562954bf72642d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a96ad39e7ee4234b562954bf72642d5. Latest consensus state: current_term: 1 leader_uuid: "8a96ad39e7ee4234b562954bf72642d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a96ad39e7ee4234b562954bf72642d5" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 37853 } } }
I20260929 10:59:34.206928 32732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a96ad39e7ee4234b562954bf72642d5 [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:34.206868 32734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a96ad39e7ee4234b562954bf72642d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a96ad39e7ee4234b562954bf72642d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a96ad39e7ee4234b562954bf72642d5" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 37853 } } }
I20260929 10:59:34.207494 32734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a96ad39e7ee4234b562954bf72642d5 [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:34.207864 32738 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:59:34.208549 32738 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:59:34.210120 32738 catalog_manager.cc:1347] Generated new cluster ID: 17f97a230f454d9481d77d546a0b5468
I20260929 10:59:34.210170 32738 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:59:34.210384 32730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:34.198848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56522 (local address 127.26.120.254:37853)
0929 10:59:34.199078 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:34.199088 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:34.199123 (+ 35us) server_negotiation.cc:407] Connection header received
0929 10:59:34.199656 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:34.199666 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:34.199727 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:34.199887 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:34.200489 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:34.201856 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:34.202738 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:34.203392 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:34.203505 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:34.206091 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:34.206096 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:34.206097 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:34.206110 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:34.208552 (+ 2442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:34.208938 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:34.208942 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:34.208943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:34.209036 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:34.209246 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:34.209247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:34.209248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:34.209540 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:34.209701 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:34.210034 (+ 333us) server_negotiation.cc:299] Negotiation successful
0929 10:59:34.210195 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":83,"threads_started":1}
I20260929 10:59:34.222781 32738 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:59:34.223430 32738 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:59:34.228904 32738 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a96ad39e7ee4234b562954bf72642d5: Generated new TSK 0
I20260929 10:59:34.229102 32738 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.120.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab.
Entry for principal kudu/127.26.120.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab.
I20260929 10:59:34.262567 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.193:0
--local_ip_for_outbound_sockets=127.26.120.193
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:37853
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:43963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:34.341589 32753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:34.341842 32753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:34.341879 32753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:34.343062 32753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:34.343135 32753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:34.343180 32753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.193
I20260929 10:59:34.344638 32753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:43963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:37853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.193
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:34.344916 32753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:34.345153 32753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:34.349988 32759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:34.349968 32761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:34.349965 32758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:34.350175 32753 server_base.cc:1034] running on GCE node
I20260929 10:59:34.350519 32753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:34.350757 32753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:34.351897 32753 hybrid_clock.cc:648] HybridClock initialized: now 1790679574351876 us; error 37 us; skew 500 ppm
Sep 29 10:59:34 dist-test-slave-2lzq krb5kdc[32646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679574, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:34.354475 32753 init.cc:377] Logged in from keytab as kudu/127.26.120.193@KRBTEST.COM (short username kudu)
I20260929 10:59:34.354888 32753 webserver.cc:460] Webserver started at http://127.26.120.193:43529/ using document root <none> and password file <none>
I20260929 10:59:34.355111 32753 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:34.355178 32753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:34.355324 32753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:34.356341 32753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-0/data/instance:
uuid: "05f163e3f45d4f14860ad2ba5434eb77"
format_stamp: "Formatted at 2026-09-29 10:59:34 on dist-test-slave-2lzq"
server_key: "8658b89a694c3f27a4ea4711d1fcc16f"
server_key_iv: "3b3040d461053220ecb767e1c00049cf"
server_key_version: "encryptionkey@0"
I20260929 10:59:34.356700 32753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance:
uuid: "05f163e3f45d4f14860ad2ba5434eb77"
format_stamp: "Formatted at 2026-09-29 10:59:34 on dist-test-slave-2lzq"
server_key: "8658b89a694c3f27a4ea4711d1fcc16f"
server_key_iv: "3b3040d461053220ecb767e1c00049cf"
server_key_version: "encryptionkey@0"
I20260929 10:59:34.358134 32753 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260929 10:59:34.358870 300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:34.359071 32753 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:59:34.359169 32753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-0/wal
uuid: "05f163e3f45d4f14860ad2ba5434eb77"
format_stamp: "Formatted at 2026-09-29 10:59:34 on dist-test-slave-2lzq"
server_key: "8658b89a694c3f27a4ea4711d1fcc16f"
server_key_iv: "3b3040d461053220ecb767e1c00049cf"
server_key_version: "encryptionkey@0"
I20260929 10:59:34.359254 32753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:34.377424 32753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:34.379781 32753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:34.379945 32753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:34.380179 32753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:59:34.380481 32753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:34.380532 32753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:34.380566 32753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:34.380594 32753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:34.386482 32753 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.193:43365
I20260929 10:59:34.386555 413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.193:43365 every 8 connection(s)
I20260929 10:59:34.386876 32753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
I20260929 10:59:34.387481 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 32753
I20260929 10:59:34.387599 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance
I20260929 10:59:34.387813 27107 external_mini_cluster.cc:1428] Setting key ac7292b04366150d8ec06d3bfbd6eb45
Sep 29 10:59:34 dist-test-slave-2lzq krb5kdc[32646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679574, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.120.194@KRBTEST.COM; defaulting to no policy
I20260929 10:59:34.397647 32730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:34.387625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:38829 (local address 127.26.120.254:37853)
0929 10:59:34.387845 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:34.387847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:34.387950 (+ 103us) server_negotiation.cc:407] Connection header received
0929 10:59:34.389221 (+ 1271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:34.389221 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:34.389231 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:34.389271 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:34.390132 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:34.390954 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:34.392033 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:34.392423 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:34.392515 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:34.394708 (+ 2193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:34.394709 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:34.394710 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:34.394719 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:34.396350 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:34.396740 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:34.396742 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:34.396742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:34.396812 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:34.397079 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:34.397080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:34.397080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:34.397195 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:34.397253 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:34.397504 (+ 251us) server_negotiation.cc:299] Negotiation successful
0929 10:59:34.397579 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197}
I20260929 10:59:34.397790 416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:34.387633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:37853 (local address 127.26.120.193:38829)
0929 10:59:34.387812 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:34.387863 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:34.389113 (+ 1250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:34.389341 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:34.389345 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:34.389731 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:34.390023 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:34.390030 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:34.391070 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:34.391071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:34.391929 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:34.391932 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:34.392530 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:34.392531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:34.392641 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:34.393208 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:34.393217 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:34.394607 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:34.396431 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:34.396437 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:34.396438 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:34.396655 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:34.396888 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:34.396889 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:34.396889 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:34.396993 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:34.397255 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:34.397259 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:34.397387 (+ 128us) client_negotiation.cc:769] Sending connection context
0929 10:59:34.397470 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 10:59:34.397597 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":80,"threads_started":1}
I20260929 10:59:34.398389 414 heartbeater.cc:347] Connected to a master server at 127.26.120.254:37853
I20260929 10:59:34.398476 414 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:34.398694 414 heartbeater.cc:511] Master 127.26.120.254:37853 requested a full tablet report, sending...
I20260929 10:59:34.399274 32691 ts_manager.cc:194] Registered new tserver with Master: 05f163e3f45d4f14860ad2ba5434eb77 (127.26.120.193:43365)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.194@KRBTEST.COM" created.
I20260929 10:59:34.399981 32691 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.193@KRBTEST.COM'} at 127.26.120.193:38829
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab.
Entry for principal kudu/127.26.120.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab.
I20260929 10:59:34.417121 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.194:0
--local_ip_for_outbound_sockets=127.26.120.194
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:37853
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:43963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:34.498976 419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:34.499190 419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:34.499213 419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:34.500401 419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:34.500447 419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:34.500506 419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.194
I20260929 10:59:34.502123 419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:43963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:37853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.194
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:34.502380 419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:34.502625 419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 10:59:34.507581 419 server_base.cc:1034] running on GCE node
W20260929 10:59:34.507644 427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:34.507622 425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:34.507686 424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:34.508010 419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:34.508250 419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:34.509469 419 hybrid_clock.cc:648] HybridClock initialized: now 1790679574509457 us; error 29 us; skew 500 ppm
Sep 29 10:59:34 dist-test-slave-2lzq krb5kdc[32646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679574, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:34.511958 419 init.cc:377] Logged in from keytab as kudu/127.26.120.194@KRBTEST.COM (short username kudu)
I20260929 10:59:34.512420 419 webserver.cc:460] Webserver started at http://127.26.120.194:34213/ using document root <none> and password file <none>
I20260929 10:59:34.512656 419 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:34.512702 419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:34.512857 419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:34.513882 419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-1/data/instance:
uuid: "dd204594be1f4131b92c453558d32dd7"
format_stamp: "Formatted at 2026-09-29 10:59:34 on dist-test-slave-2lzq"
server_key: "ee751505240ba3141170a4fb08e3038d"
server_key_iv: "a6a5be5e0112ccdf6ac392a723918237"
server_key_version: "encryptionkey@0"
I20260929 10:59:34.514271 419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance:
uuid: "dd204594be1f4131b92c453558d32dd7"
format_stamp: "Formatted at 2026-09-29 10:59:34 on dist-test-slave-2lzq"
server_key: "ee751505240ba3141170a4fb08e3038d"
server_key_iv: "a6a5be5e0112ccdf6ac392a723918237"
server_key_version: "encryptionkey@0"
I20260929 10:59:34.515555 419 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:59:34.516250 434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:34.516434 419 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:59:34.516526 419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-1/wal
uuid: "dd204594be1f4131b92c453558d32dd7"
format_stamp: "Formatted at 2026-09-29 10:59:34 on dist-test-slave-2lzq"
server_key: "ee751505240ba3141170a4fb08e3038d"
server_key_iv: "a6a5be5e0112ccdf6ac392a723918237"
server_key_version: "encryptionkey@0"
I20260929 10:59:34.516610 419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:34.528919 419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:34.530804 419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:34.530961 419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:34.531190 419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:59:34.531512 419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:34.531558 419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:34.531605 419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:34.531631 419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:34.537551 419 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.194:38473
I20260929 10:59:34.537583 547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.194:38473 every 8 connection(s)
I20260929 10:59:34.537925 419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
I20260929 10:59:34.542371 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 419
I20260929 10:59:34.542474 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance
I20260929 10:59:34.542662 27107 external_mini_cluster.cc:1428] Setting key c45f3f2f0e21893e3b5a8ed122c929a7
Sep 29 10:59:34 dist-test-slave-2lzq krb5kdc[32646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679574, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:34.548561 32730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:34.538846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:43243 (local address 127.26.120.254:37853)
0929 10:59:34.538923 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:34.538926 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:34.539143 (+ 217us) server_negotiation.cc:407] Connection header received
0929 10:59:34.540096 (+ 953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:34.540097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:34.540106 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:34.540163 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:34.540786 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:34.541276 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:34.542398 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:34.542669 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:34.542755 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:34.545301 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:34.545303 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:34.545304 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:34.545312 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:34.547066 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:34.547464 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:34.547466 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:34.547467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:34.547544 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:34.547803 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:34.547804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:34.547804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:34.547971 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:34.548046 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:34.548315 (+ 269us) server_negotiation.cc:299] Negotiation successful
0929 10:59:34.548413 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 10:59:34.548580 550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:34.538822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:37853 (local address 127.26.120.194:43243)
0929 10:59:34.539011 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:34.539049 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:34.539994 (+ 945us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:34.540240 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:34.540244 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:34.540518 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:34.540710 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:34.540713 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:34.541394 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:34.541394 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:34.542301 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:34.542303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:34.542781 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:34.542783 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:34.542935 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:34.543520 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:34.543531 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:34.545198 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:34.547159 (+ 1961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:34.547165 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:34.547167 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:34.547401 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:34.547631 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:34.547632 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:34.547632 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:34.547734 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:34.548041 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:34.548045 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:34.548169 (+ 124us) client_negotiation.cc:769] Sending connection context
0929 10:59:34.548258 (+ 89us) client_negotiation.cc:240] Negotiation successful
0929 10:59:34.548392 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":100,"threads_started":1}
I20260929 10:59:34.549095 548 heartbeater.cc:347] Connected to a master server at 127.26.120.254:37853
I20260929 10:59:34.549213 548 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:34.549422 548 heartbeater.cc:511] Master 127.26.120.254:37853 requested a full tablet report, sending...
I20260929 10:59:34.549882 32691 ts_manager.cc:194] Registered new tserver with Master: dd204594be1f4131b92c453558d32dd7 (127.26.120.194:38473)
I20260929 10:59:34.550351 32691 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.194@KRBTEST.COM'} at 127.26.120.194:43243
WARNING: no policy specified for kudu/127.26.120.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab.
Entry for principal kudu/127.26.120.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab.
I20260929 10:59:34.569538 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.195:0
--local_ip_for_outbound_sockets=127.26.120.195
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:37853
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:43963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:34.649783 553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:34.649983 553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:34.650007 553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:34.651161 553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:34.651202 553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:34.651228 553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.195
I20260929 10:59:34.652601 553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:43963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:37853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.195
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:34.652772 553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:34.652992 553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:34.657835 561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:34.657972 558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:34.657830 559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:34.658115 553 server_base.cc:1034] running on GCE node
I20260929 10:59:34.658342 553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:34.658604 553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:34.659759 553 hybrid_clock.cc:648] HybridClock initialized: now 1790679574659745 us; error 31 us; skew 500 ppm
Sep 29 10:59:34 dist-test-slave-2lzq krb5kdc[32646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679574, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:34.662192 553 init.cc:377] Logged in from keytab as kudu/127.26.120.195@KRBTEST.COM (short username kudu)
I20260929 10:59:34.662680 553 webserver.cc:460] Webserver started at http://127.26.120.195:33469/ using document root <none> and password file <none>
I20260929 10:59:34.662909 553 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:34.662976 553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:34.663110 553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:34.664214 553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-2/data/instance:
uuid: "722654f6f6de43809b92aba394779923"
format_stamp: "Formatted at 2026-09-29 10:59:34 on dist-test-slave-2lzq"
server_key: "2f116e69ffab865433f0456baf591681"
server_key_iv: "f299af1fb106ca5c2a530677d546b504"
server_key_version: "encryptionkey@0"
I20260929 10:59:34.664590 553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance:
uuid: "722654f6f6de43809b92aba394779923"
format_stamp: "Formatted at 2026-09-29 10:59:34 on dist-test-slave-2lzq"
server_key: "2f116e69ffab865433f0456baf591681"
server_key_iv: "f299af1fb106ca5c2a530677d546b504"
server_key_version: "encryptionkey@0"
I20260929 10:59:34.665871 553 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:59:34.666643 568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:34.666805 553 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:34.666899 553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-2/wal
uuid: "722654f6f6de43809b92aba394779923"
format_stamp: "Formatted at 2026-09-29 10:59:34 on dist-test-slave-2lzq"
server_key: "2f116e69ffab865433f0456baf591681"
server_key_iv: "f299af1fb106ca5c2a530677d546b504"
server_key_version: "encryptionkey@0"
I20260929 10:59:34.666975 553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:34.704264 553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:34.706274 553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:34.706435 553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:34.706717 553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:59:34.707037 553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:34.707091 553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:34.707132 553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:34.707160 553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:34.713186 553 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.195:39191
I20260929 10:59:34.713229 681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.195:39191 every 8 connection(s)
I20260929 10:59:34.713546 553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
I20260929 10:59:34.714546 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 553
I20260929 10:59:34.714639 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance
I20260929 10:59:34.714818 27107 external_mini_cluster.cc:1428] Setting key 053b4443d581ac7e19da6f4185733cab
Sep 29 10:59:34 dist-test-slave-2lzq krb5kdc[32646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679574, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:34.722874 32730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:34.714325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:51325 (local address 127.26.120.254:37853)
0929 10:59:34.714386 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:34.714404 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:34.714627 (+ 223us) server_negotiation.cc:407] Connection header received
0929 10:59:34.715761 (+ 1134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:34.715762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:34.715771 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:34.715816 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:34.716390 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:34.716873 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:34.717837 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:34.718026 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:34.718134 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:34.720324 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:34.720326 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:34.720326 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:34.720333 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:34.721709 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:34.722096 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:34.722098 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:34.722098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:34.722154 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:34.722371 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:34.722372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:34.722372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:34.722471 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:34.722524 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:34.722753 (+ 229us) server_negotiation.cc:299] Negotiation successful
0929 10:59:34.722812 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260929 10:59:34.723099 684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:34.714319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:37853 (local address 127.26.120.195:51325)
0929 10:59:34.714488 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:34.714531 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:34.715683 (+ 1152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:34.715873 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:34.715877 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:34.716127 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:34.716321 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:34.716324 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:34.716950 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:34.716950 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:34.717765 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:34.717767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:34.718139 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:34.718140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:34.718252 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:34.718700 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:34.718708 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:34.720210 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:34.721774 (+ 1564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:34.721779 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:34.721781 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:34.721989 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:34.722213 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:34.722213 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:34.722213 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:34.722312 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:34.722524 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:34.722529 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:34.722645 (+ 116us) client_negotiation.cc:769] Sending connection context
0929 10:59:34.722728 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 10:59:34.722843 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":98,"threads_started":1}
I20260929 10:59:34.723588 682 heartbeater.cc:347] Connected to a master server at 127.26.120.254:37853
I20260929 10:59:34.723667 682 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:34.723941 682 heartbeater.cc:511] Master 127.26.120.254:37853 requested a full tablet report, sending...
I20260929 10:59:34.724363 32691 ts_manager.cc:194] Registered new tserver with Master: 722654f6f6de43809b92aba394779923 (127.26.120.195:39191)
I20260929 10:59:34.724738 32691 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.195@KRBTEST.COM'} at 127.26.120.195:51325
I20260929 10:59:34.726356 27107 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/krb5kdc/test-admin.keytab.
Sep 29 10:59:34 dist-test-slave-2lzq krb5kdc[32646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:34.742443 27107 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260929 10:59:34.744014 689 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:34.743226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34400 (local address 127.26.120.193:43365)
0929 10:59:34.743399 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:34.743405 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:34.743418 (+ 13us) server_negotiation.cc:407] Connection header received
0929 10:59:34.743442 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:34.743444 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:34.743457 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:34.743547 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:34.743902 (+ 355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34400: BlockingRecv error: recv got EOF from 127.0.0.1:34400 (error 108)
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":77,"threads_started":1}
W20260929 10:59:34.744118 27107 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/krb5kdc/krb5cc' of type FILE
Sep 29 10:59:34 dist-test-slave-2lzq krb5kdc[32646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:34.745064 27107 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 10:59:34 dist-test-slave-2lzq krb5kdc[32646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:34.745903 27107 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 10:59:34 dist-test-slave-2lzq krb5kdc[32646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.193@KRBTEST.COM
I20260929 10:59:34.753410 689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:34.746431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34406 (local address 127.26.120.193:43365)
0929 10:59:34.746484 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:34.746486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:34.746497 (+ 11us) server_negotiation.cc:407] Connection header received
0929 10:59:34.746549 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:34.746549 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:34.746559 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:34.746609 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:34.747071 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:34.747771 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:34.748456 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:34.748768 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:34.748820 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:34.750864 (+ 2044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:34.750872 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:34.750872 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:34.750879 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:34.752471 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:34.752774 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:34.752777 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:34.752777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:34.752828 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:34.752982 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:34.752983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:34.752983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:34.753125 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:34.753246 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:34.753284 (+ 38us) server_negotiation.cc:299] Negotiation successful
0929 10:59:34.753341 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260929 10:59:34.755314 689 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:34.754758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34414 (local address 127.26.120.193:43365)
0929 10:59:34.754807 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:34.754809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:34.754860 (+ 51us) server_negotiation.cc:407] Connection header received
0929 10:59:34.754931 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:34.754932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:34.754942 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:34.754984 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:34.755265 (+ 281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34414: BlockingRecv error: recv got EOF from 127.0.0.1:34414 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260929 10:59:34.755380 27107 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/krb5kdc/krb5cc' of type FILE
Sep 29 10:59:34 dist-test-slave-2lzq krb5kdc[32646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:34.756328 27107 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 10:59:34 dist-test-slave-2lzq krb5kdc[32646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:34.757139 27107 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 10:59:34 dist-test-slave-2lzq krb5kdc[32646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.193@KRBTEST.COM
I20260929 10:59:34.763935 689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:34.757592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34418 (local address 127.26.120.193:43365)
0929 10:59:34.757627 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:34.757629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:34.757714 (+ 85us) server_negotiation.cc:407] Connection header received
0929 10:59:34.757771 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:34.757771 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:34.757782 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:34.757831 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:34.758239 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:34.758711 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:34.759403 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:34.759609 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:34.759692 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:34.761561 (+ 1869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:34.761562 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:34.761562 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:34.761570 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:34.762884 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:34.763217 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:34.763219 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:34.763219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:34.763270 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:34.763467 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:34.763468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:34.763468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:34.763568 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:34.763667 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:34.763771 (+ 104us) server_negotiation.cc:299] Negotiation successful
0929 10:59:34.763845 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260929 10:59:34.765267 689 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:34.764818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34426 (local address 127.26.120.193:43365)
0929 10:59:34.764872 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:34.764874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:34.764883 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:59:34.764932 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:34.764933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:34.764940 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:34.764992 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:34.765206 (+ 214us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34426: BlockingRecv error: recv got EOF from 127.0.0.1:34426 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260929 10:59:34.765347 27107 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790679549986039-27107-0/krb5kdc/krb5cc' of type FILE
Sep 29 10:59:34 dist-test-slave-2lzq krb5kdc[32646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:34.766309 27107 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 10:59:34 dist-test-slave-2lzq krb5kdc[32646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:34.767143 27107 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 10:59:34 dist-test-slave-2lzq krb5kdc[32646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.193@KRBTEST.COM
I20260929 10:59:34.773711 689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:34.767646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34428 (local address 127.26.120.193:43365)
0929 10:59:34.767682 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:34.767684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:34.767713 (+ 29us) server_negotiation.cc:407] Connection header received
0929 10:59:34.767784 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:34.767785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:34.767792 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:34.767837 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:34.768151 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:34.768598 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:34.769287 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:34.769475 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:34.769524 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:34.771511 (+ 1987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:34.771512 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:34.771512 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:34.771518 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:34.772830 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:34.773100 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:34.773100 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:34.773101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:34.773147 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:34.773290 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:34.773291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:34.773291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:34.773365 (+ 74us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:34.773426 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:34.773506 (+ 80us) server_negotiation.cc:299] Negotiation successful
0929 10:59:34.773567 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 10:59:34.774326 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 32753
I20260929 10:59:34.780265 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 419
I20260929 10:59:34.784323 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 553
I20260929 10:59:34.788373 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 32659
2026-09-29T10:59:34Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1854 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:59:34 dist-test-slave-2lzq krb5kdc[702](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:59:34 dist-test-slave-2lzq krb5kdc[702](info): set up 2 sockets
Sep 29 10:59:34 dist-test-slave-2lzq krb5kdc[702](info): commencing operation
krb5kdc: starting...
W20260929 10:59:35.822592 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 10:59:35 dist-test-slave-2lzq krb5kdc[702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:59:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:59:35Z Disabled control of system clock
WARNING: no policy specified for oryx/127.26.120.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.120.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.120.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.254.keytab.
Entry for principal oryx/127.26.120.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.254.keytab.
I20260929 10:59:35.981108 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:34785
--webserver_interface=127.26.120.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.254.keytab
--principal=oryx/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:41445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:34785
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:36.060218 715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:36.060452 715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:36.060477 715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:36.061640 715 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:59:36.061681 715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:36.061695 715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:36.061708 715 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:59:36.061720 715 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:59:36.063225 715 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:41445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:34785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.254.keytab
--principal=oryx/127.26.120.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:34785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:36.063479 715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:36.063705 715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:36.068643 720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:36.068614 723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:36.068624 721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:36.068863 715 server_base.cc:1034] running on GCE node
I20260929 10:59:36.069082 715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:36.069372 715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:36.070568 715 hybrid_clock.cc:648] HybridClock initialized: now 1790679576070553 us; error 26 us; skew 500 ppm
Sep 29 10:59:36 dist-test-slave-2lzq krb5kdc[702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679576, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:36.072906 715 init.cc:377] Logged in from keytab as oryx/127.26.120.254@KRBTEST.COM (short username oryx)
I20260929 10:59:36.073354 715 webserver.cc:460] Webserver started at http://127.26.120.254:36451/ using document root <none> and password file <none>
I20260929 10:59:36.073570 715 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:36.073634 715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:36.073787 715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:36.074877 715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d"
format_stamp: "Formatted at 2026-09-29 10:59:36 on dist-test-slave-2lzq"
server_key: "0f86bf0105652e141fe929bb48d2c838"
server_key_iv: "bf8edbaa3a421ba12f5a7dbfa14b2cf2"
server_key_version: "encryptionkey@0"
I20260929 10:59:36.075235 715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d"
format_stamp: "Formatted at 2026-09-29 10:59:36 on dist-test-slave-2lzq"
server_key: "0f86bf0105652e141fe929bb48d2c838"
server_key_iv: "bf8edbaa3a421ba12f5a7dbfa14b2cf2"
server_key_version: "encryptionkey@0"
I20260929 10:59:36.076526 715 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 10:59:36.077226 730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:36.077415 715 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:59:36.077513 715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d"
format_stamp: "Formatted at 2026-09-29 10:59:36 on dist-test-slave-2lzq"
server_key: "0f86bf0105652e141fe929bb48d2c838"
server_key_iv: "bf8edbaa3a421ba12f5a7dbfa14b2cf2"
server_key_version: "encryptionkey@0"
I20260929 10:59:36.077593 715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:36.098310 715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:36.100274 715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:36.100448 715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:36.104746 715 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:34785
I20260929 10:59:36.104782 782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:34785 every 8 connection(s)
I20260929 10:59:36.105154 715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 10:59:36.105909 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 715
I20260929 10:59:36.106025 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 10:59:36.106261 27107 external_mini_cluster.cc:1428] Setting key 25ac952b2f4f043e35c3039162f8e212
I20260929 10:59:36.106375 783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:36.110240 783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d: Bootstrap starting.
I20260929 10:59:36.110873 783 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:36.111230 783 log.cc:824] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:36.111999 783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d: No bootstrap required, opened a new log
Sep 29 10:59:36 dist-test-slave-2lzq krb5kdc[702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM
I20260929 10:59:36.114235 783 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } }
I20260929 10:59:36.114423 783 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:36.114472 783 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c9a8ea9e35a43a99072b3ab64ca4b8d, State: Initialized, Role: FOLLOWER
I20260929 10:59:36.114603 783 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } }
I20260929 10:59:36.114698 783 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:59:36.114737 783 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:59:36.114794 783 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:36.115567 783 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } }
I20260929 10:59:36.115680 783 leader_election.cc:302] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c9a8ea9e35a43a99072b3ab64ca4b8d; no voters:
I20260929 10:59:36.115839 783 leader_election.cc:288] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:59:36.115913 788 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:59:36.116101 788 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [term 1 LEADER]: Becoming Leader. State: Replica: 5c9a8ea9e35a43a99072b3ab64ca4b8d, State: Running, Role: LEADER
I20260929 10:59:36.116114 783 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:59:36.116206 788 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } }
I20260929 10:59:36.116547 789 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c9a8ea9e35a43a99072b3ab64ca4b8d. Latest consensus state: current_term: 1 leader_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } }
I20260929 10:59:36.116647 789 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:36.116859 795 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:59:36.117406 795 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:59:36.116837 790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } }
I20260929 10:59:36.118171 790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:36.118418 786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:36.106826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60486 (local address 127.26.120.254:34785)
0929 10:59:36.107011 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:36.107054 (+ 43us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:36.107091 (+ 37us) server_negotiation.cc:407] Connection header received
0929 10:59:36.107908 (+ 817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:36.107918 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:36.107999 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:36.108172 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:36.108885 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.110331 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.111156 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.111766 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.111879 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:36.114123 (+ 2244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:36.114127 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:36.114128 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:36.114140 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:36.116277 (+ 2137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:36.116739 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:36.116743 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:36.116743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:36.116866 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:36.117110 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:36.117111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:36.117111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:36.117435 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:36.117579 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:36.118032 (+ 453us) server_negotiation.cc:299] Negotiation successful
0929 10:59:36.118235 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":95,"threads_started":1}
I20260929 10:59:36.119028 795 catalog_manager.cc:1347] Generated new cluster ID: 27ec5d70879346ffba3bed1fd19d6dff
I20260929 10:59:36.119076 795 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:59:36.144805 795 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:59:36.145545 795 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:59:36.161379 795 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d: Generated new TSK 0
I20260929 10:59:36.161592 795 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.26.120.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.120.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.120.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.193.keytab.
Entry for principal oryx/127.26.120.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.193.keytab.
I20260929 10:59:36.192910 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.193:0
--local_ip_for_outbound_sockets=127.26.120.193
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:34785
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.193.keytab
--principal=oryx/127.26.120.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:41445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:36.299088 809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:36.299294 809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:36.299317 809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:36.300514 809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:36.300561 809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:36.300588 809 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.193
I20260929 10:59:36.302045 809 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:41445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.193.keytab
--principal=oryx/127.26.120.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:34785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.193
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:36.302312 809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:36.302567 809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:36.307626 817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:36.307669 809 server_base.cc:1034] running on GCE node
W20260929 10:59:36.307601 814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:36.307621 815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:36.308025 809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:36.308274 809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:36.309450 809 hybrid_clock.cc:648] HybridClock initialized: now 1790679576309439 us; error 31 us; skew 500 ppm
Sep 29 10:59:36 dist-test-slave-2lzq krb5kdc[702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679576, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:36.311899 809 init.cc:377] Logged in from keytab as oryx/127.26.120.193@KRBTEST.COM (short username oryx)
I20260929 10:59:36.312330 809 webserver.cc:460] Webserver started at http://127.26.120.193:41851/ using document root <none> and password file <none>
I20260929 10:59:36.312574 809 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:36.312644 809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:36.312794 809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:36.313832 809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-0/data/instance:
uuid: "deddd697abc04e13b689f98668eb1250"
format_stamp: "Formatted at 2026-09-29 10:59:36 on dist-test-slave-2lzq"
server_key: "be50769201a49c5adc7efc41112ff0dd"
server_key_iv: "78d7fb123a3f7b8b98839496102c1a87"
server_key_version: "encryptionkey@0"
I20260929 10:59:36.314245 809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance:
uuid: "deddd697abc04e13b689f98668eb1250"
format_stamp: "Formatted at 2026-09-29 10:59:36 on dist-test-slave-2lzq"
server_key: "be50769201a49c5adc7efc41112ff0dd"
server_key_iv: "78d7fb123a3f7b8b98839496102c1a87"
server_key_version: "encryptionkey@0"
I20260929 10:59:36.315562 809 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:59:36.316329 824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:36.316493 809 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:59:36.316572 809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-0/wal
uuid: "deddd697abc04e13b689f98668eb1250"
format_stamp: "Formatted at 2026-09-29 10:59:36 on dist-test-slave-2lzq"
server_key: "be50769201a49c5adc7efc41112ff0dd"
server_key_iv: "78d7fb123a3f7b8b98839496102c1a87"
server_key_version: "encryptionkey@0"
I20260929 10:59:36.316627 809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:36.326942 809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:36.328850 809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:36.328963 809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:36.329139 809 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:59:36.329469 809 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:36.329509 809 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:36.329556 809 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:36.329582 809 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:36.335337 809 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.193:42371
I20260929 10:59:36.335402 937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.193:42371 every 8 connection(s)
I20260929 10:59:36.335728 809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
I20260929 10:59:36.338353 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 809
I20260929 10:59:36.338428 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance
I20260929 10:59:36.338620 27107 external_mini_cluster.cc:1428] Setting key 947a5cb82b8eb670f654d66b3b05daf7
Sep 29 10:59:36 dist-test-slave-2lzq krb5kdc[702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679576, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM
I20260929 10:59:36.346472 786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:36.336417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:51755 (local address 127.26.120.254:34785)
0929 10:59:36.336461 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:36.336463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:36.336652 (+ 189us) server_negotiation.cc:407] Connection header received
0929 10:59:36.337134 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:36.337135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:36.337145 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:36.337197 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:36.338204 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.338867 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.340351 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.340541 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.340595 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:36.343201 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:36.343203 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:36.343203 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:36.343211 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:36.344928 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:36.345367 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:36.345368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:36.345369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:36.345437 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:36.345683 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:36.345684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:36.345685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:36.345839 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:36.345919 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:36.346276 (+ 357us) server_negotiation.cc:299] Negotiation successful
0929 10:59:36.346373 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 10:59:36.346565 940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:36.336406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34785 (local address 127.26.120.193:51755)
0929 10:59:36.336549 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:36.336584 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:36.337046 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:36.337250 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:36.337253 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:36.337845 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:36.338099 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:36.338108 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.338968 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.338969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:36.340255 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:36.340259 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.340624 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.340625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:36.340783 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:36.341353 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:36.341362 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:36.343093 (+ 1731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:36.345021 (+ 1928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:36.345028 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:36.345030 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:36.345285 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:36.345509 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:36.345509 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:36.345509 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:36.345612 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:36.345913 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:36.345919 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:36.346050 (+ 131us) client_negotiation.cc:769] Sending connection context
0929 10:59:36.346230 (+ 180us) client_negotiation.cc:240] Negotiation successful
0929 10:59:36.346365 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":86,"threads_started":1}
I20260929 10:59:36.347121 938 heartbeater.cc:347] Connected to a master server at 127.26.120.254:34785
I20260929 10:59:36.347214 938 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for oryx/127.26.120.194@KRBTEST.COM; defaulting to no policy
I20260929 10:59:36.347400 938 heartbeater.cc:511] Master 127.26.120.254:34785 requested a full tablet report, sending...
I20260929 10:59:36.347966 747 ts_manager.cc:194] Registered new tserver with Master: deddd697abc04e13b689f98668eb1250 (127.26.120.193:42371)
I20260929 10:59:36.348654 747 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.120.193@KRBTEST.COM'} at 127.26.120.193:51755
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.120.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.120.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.194.keytab.
Entry for principal oryx/127.26.120.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.194.keytab.
I20260929 10:59:36.364951 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.194:0
--local_ip_for_outbound_sockets=127.26.120.194
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:34785
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.194.keytab
--principal=oryx/127.26.120.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:41445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:36.446569 943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:36.446771 943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:36.446794 943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:36.447862 943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:36.447896 943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:36.447921 943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.194
I20260929 10:59:36.449261 943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:41445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.194.keytab
--principal=oryx/127.26.120.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:34785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.194
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:36.449456 943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:36.449623 943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:36.454550 949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:36.454619 943 server_base.cc:1034] running on GCE node
W20260929 10:59:36.454515 948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:36.454515 951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:36.454933 943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:36.455199 943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:36.456367 943 hybrid_clock.cc:648] HybridClock initialized: now 1790679576456350 us; error 49 us; skew 500 ppm
Sep 29 10:59:36 dist-test-slave-2lzq krb5kdc[702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679576, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:36.458787 943 init.cc:377] Logged in from keytab as oryx/127.26.120.194@KRBTEST.COM (short username oryx)
I20260929 10:59:36.459213 943 webserver.cc:460] Webserver started at http://127.26.120.194:42179/ using document root <none> and password file <none>
I20260929 10:59:36.459447 943 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:36.459517 943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:36.459673 943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:36.460680 943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-1/data/instance:
uuid: "b586e21fea2f46c385b5f49e297018b7"
format_stamp: "Formatted at 2026-09-29 10:59:36 on dist-test-slave-2lzq"
server_key: "1b240428a87574cd6bca1ac2e6d47121"
server_key_iv: "5ac0bcc67f374bed9794990cf5924492"
server_key_version: "encryptionkey@0"
I20260929 10:59:36.461059 943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance:
uuid: "b586e21fea2f46c385b5f49e297018b7"
format_stamp: "Formatted at 2026-09-29 10:59:36 on dist-test-slave-2lzq"
server_key: "1b240428a87574cd6bca1ac2e6d47121"
server_key_iv: "5ac0bcc67f374bed9794990cf5924492"
server_key_version: "encryptionkey@0"
I20260929 10:59:36.462395 943 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 10:59:36.463106 958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:36.463264 943 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:59:36.463359 943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-1/wal
uuid: "b586e21fea2f46c385b5f49e297018b7"
format_stamp: "Formatted at 2026-09-29 10:59:36 on dist-test-slave-2lzq"
server_key: "1b240428a87574cd6bca1ac2e6d47121"
server_key_iv: "5ac0bcc67f374bed9794990cf5924492"
server_key_version: "encryptionkey@0"
I20260929 10:59:36.463442 943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:36.475534 943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:36.477350 943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:36.477515 943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:36.477736 943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:59:36.478044 943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:36.478188 943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:36.478231 943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:36.478259 943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:36.484288 943 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.194:44103
I20260929 10:59:36.484332 1071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.194:44103 every 8 connection(s)
I20260929 10:59:36.484647 943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
I20260929 10:59:36.489732 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 943
I20260929 10:59:36.489851 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance
I20260929 10:59:36.490021 27107 external_mini_cluster.cc:1428] Setting key 310e2e02825f5ee741e030e8ccfe5b0b
Sep 29 10:59:36 dist-test-slave-2lzq krb5kdc[702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679576, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM
I20260929 10:59:36.494166 786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:36.485328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:51185 (local address 127.26.120.254:34785)
0929 10:59:36.485394 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:36.485396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:36.485583 (+ 187us) server_negotiation.cc:407] Connection header received
0929 10:59:36.486069 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:36.486070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:36.486079 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:36.486150 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:36.487044 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.487533 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.488500 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.488687 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.488738 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:36.490920 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:36.490922 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:36.490922 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:36.490931 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:36.492551 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:36.493056 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:36.493057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:36.493057 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:36.493111 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:36.493399 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:36.493400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:36.493400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:36.493503 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:36.493593 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:36.493899 (+ 306us) server_negotiation.cc:299] Negotiation successful
0929 10:59:36.493984 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 10:59:36.494175 1074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:36.485316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34785 (local address 127.26.120.194:51185)
0929 10:59:36.485456 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:36.485496 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:36.485958 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:36.486207 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:36.486211 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:36.486764 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:36.486954 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:36.486958 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.487619 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.487619 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:36.488424 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:36.488426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.488757 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.488758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:36.488858 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:36.489312 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:36.489320 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:36.490836 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:36.492642 (+ 1806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:36.492649 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:36.492652 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:36.492969 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:36.493179 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:36.493180 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:36.493180 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:36.493328 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:36.493596 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:36.493601 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:36.493754 (+ 153us) client_negotiation.cc:769] Sending connection context
0929 10:59:36.493840 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 10:59:36.493956 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":75,"threads_started":1}
I20260929 10:59:36.494668 1072 heartbeater.cc:347] Connected to a master server at 127.26.120.254:34785
I20260929 10:59:36.494776 1072 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:36.494988 1072 heartbeater.cc:511] Master 127.26.120.254:34785 requested a full tablet report, sending...
I20260929 10:59:36.495401 747 ts_manager.cc:194] Registered new tserver with Master: b586e21fea2f46c385b5f49e297018b7 (127.26.120.194:44103)
I20260929 10:59:36.495777 747 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.120.194@KRBTEST.COM'} at 127.26.120.194:51185
WARNING: no policy specified for oryx/127.26.120.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.120.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.120.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.195.keytab.
Entry for principal oryx/127.26.120.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.195.keytab.
I20260929 10:59:36.514917 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.195:0
--local_ip_for_outbound_sockets=127.26.120.195
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:34785
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.195.keytab
--principal=oryx/127.26.120.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:41445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:36.595317 1077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:36.595528 1077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:36.595561 1077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:36.596727 1077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:36.596769 1077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:36.596796 1077 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.195
I20260929 10:59:36.598323 1077 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:41445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.195.keytab
--principal=oryx/127.26.120.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:34785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.195
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:36.598574 1077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:36.598807 1077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:36.603765 1085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:36.603786 1083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:36.603940 1082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:36.604135 1077 server_base.cc:1034] running on GCE node
I20260929 10:59:36.604363 1077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:36.604602 1077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:36.605764 1077 hybrid_clock.cc:648] HybridClock initialized: now 1790679576605752 us; error 23 us; skew 500 ppm
Sep 29 10:59:36 dist-test-slave-2lzq krb5kdc[702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679576, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:36.608189 1077 init.cc:377] Logged in from keytab as oryx/127.26.120.195@KRBTEST.COM (short username oryx)
I20260929 10:59:36.608649 1077 webserver.cc:460] Webserver started at http://127.26.120.195:44259/ using document root <none> and password file <none>
I20260929 10:59:36.608866 1077 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:36.608932 1077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:36.609089 1077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:36.610175 1077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-2/data/instance:
uuid: "83065b57befe412fbf130152240e8700"
format_stamp: "Formatted at 2026-09-29 10:59:36 on dist-test-slave-2lzq"
server_key: "31bdc1a37741a1188c307956500136d2"
server_key_iv: "63b63cbb47511c82a7a3dd0be6bc9f67"
server_key_version: "encryptionkey@0"
I20260929 10:59:36.610538 1077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance:
uuid: "83065b57befe412fbf130152240e8700"
format_stamp: "Formatted at 2026-09-29 10:59:36 on dist-test-slave-2lzq"
server_key: "31bdc1a37741a1188c307956500136d2"
server_key_iv: "63b63cbb47511c82a7a3dd0be6bc9f67"
server_key_version: "encryptionkey@0"
I20260929 10:59:36.611848 1077 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:59:36.612560 1092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:36.612727 1077 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:59:36.612823 1077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-2/wal
uuid: "83065b57befe412fbf130152240e8700"
format_stamp: "Formatted at 2026-09-29 10:59:36 on dist-test-slave-2lzq"
server_key: "31bdc1a37741a1188c307956500136d2"
server_key_iv: "63b63cbb47511c82a7a3dd0be6bc9f67"
server_key_version: "encryptionkey@0"
I20260929 10:59:36.612910 1077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:36.624944 1077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:36.627017 1077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:36.627193 1077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:36.627429 1077 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:59:36.627740 1077 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:36.627796 1077 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:36.627842 1077 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:36.627871 1077 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:36.633666 1077 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.195:43039
I20260929 10:59:36.633713 1205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.195:43039 every 8 connection(s)
I20260929 10:59:36.634091 1077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
I20260929 10:59:36.639672 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 1077
I20260929 10:59:36.639791 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance
I20260929 10:59:36.639961 27107 external_mini_cluster.cc:1428] Setting key 1b97eb895d6b8b32a61a537c7a2b1cf8
Sep 29 10:59:36 dist-test-slave-2lzq krb5kdc[702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679576, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM
I20260929 10:59:36.643863 786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:36.634803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:46163 (local address 127.26.120.254:34785)
0929 10:59:36.634863 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:36.634865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:36.635069 (+ 204us) server_negotiation.cc:407] Connection header received
0929 10:59:36.635597 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:36.635599 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:36.635610 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:36.635680 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:36.636608 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.637113 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.638171 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.638374 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.638461 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:36.640936 (+ 2475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:36.640937 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:36.640938 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:36.640945 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:36.642455 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:36.642889 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:36.642890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:36.642891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:36.642949 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:36.643213 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:36.643214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:36.643215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:36.643332 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:36.643405 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:36.643691 (+ 286us) server_negotiation.cc:299] Negotiation successful
0929 10:59:36.643770 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260929 10:59:36.644071 1208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:36.634776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34785 (local address 127.26.120.195:46163)
0929 10:59:36.634952 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:36.634986 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:36.635478 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:36.635749 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:36.635753 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:36.636313 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:36.636496 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:36.636509 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.637208 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.637209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:36.638027 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:36.638029 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.638470 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.638471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:36.638601 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:36.639146 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:36.639154 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:36.640850 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:36.642557 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:36.642564 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:36.642567 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:36.642794 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:36.643017 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:36.643018 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:36.643019 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:36.643137 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:36.643407 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:36.643414 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:36.643558 (+ 144us) client_negotiation.cc:769] Sending connection context
0929 10:59:36.643672 (+ 114us) client_negotiation.cc:240] Negotiation successful
0929 10:59:36.643833 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":87,"threads_started":1}
I20260929 10:59:36.644531 1206 heartbeater.cc:347] Connected to a master server at 127.26.120.254:34785
I20260929 10:59:36.644613 1206 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:36.644747 1206 heartbeater.cc:511] Master 127.26.120.254:34785 requested a full tablet report, sending...
I20260929 10:59:36.645095 747 ts_manager.cc:194] Registered new tserver with Master: 83065b57befe412fbf130152240e8700 (127.26.120.195:43039)
I20260929 10:59:36.645473 747 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.120.195@KRBTEST.COM'} at 127.26.120.195:46163
I20260929 10:59:36.651839 27107 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.26.120.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.120.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.120.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.253.keytab.
Entry for principal oryx/127.26.120.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.253.keytab.
I20260929 10:59:36.676914 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.253:33873
--webserver_interface=127.26.120.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.253.keytab
--principal=oryx/127.26.120.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:41445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:34785,127.26.120.253:33873
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:36.759065 1211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:36.759277 1211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:36.759299 1211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:36.760499 1211 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:59:36.760546 1211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:36.760560 1211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:36.760573 1211 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:59:36.760586 1211 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:59:36.762212 1211 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:41445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:34785,127.26.120.253:33873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.253.keytab
--principal=oryx/127.26.120.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.253:33873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.120.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
W20260929 10:59:36.762352 1211 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.26.120.254:34785,127.26.120.253:33873'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260929 10:59:36.762496 1211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:36.762727 1211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:36.767675 1219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:36.767635 1217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:36.767655 1211 server_base.cc:1034] running on GCE node
W20260929 10:59:36.767666 1216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:36.768115 1211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:36.768420 1211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:36.769589 1211 hybrid_clock.cc:648] HybridClock initialized: now 1790679576769581 us; error 47 us; skew 500 ppm
Sep 29 10:59:36 dist-test-slave-2lzq krb5kdc[702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679576, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:36.772130 1211 init.cc:377] Logged in from keytab as oryx/127.26.120.253@KRBTEST.COM (short username oryx)
I20260929 10:59:36.772598 1211 webserver.cc:460] Webserver started at http://127.26.120.253:43941/ using document root <none> and password file <none>
I20260929 10:59:36.772831 1211 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:36.772902 1211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:36.773054 1211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:36.774227 1211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-1/data/instance:
uuid: "2db81c82699747b3bd5bef1172282f3b"
format_stamp: "Formatted at 2026-09-29 10:59:36 on dist-test-slave-2lzq"
server_key: "c251c0a857b34b510e4a50922cdbc191"
server_key_iv: "7f98bbbc6a690a207761915e3b3e5e22"
server_key_version: "encryptionkey@0"
I20260929 10:59:36.774601 1211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-1/wal/instance:
uuid: "2db81c82699747b3bd5bef1172282f3b"
format_stamp: "Formatted at 2026-09-29 10:59:36 on dist-test-slave-2lzq"
server_key: "c251c0a857b34b510e4a50922cdbc191"
server_key_iv: "7f98bbbc6a690a207761915e3b3e5e22"
server_key_version: "encryptionkey@0"
I20260929 10:59:36.775918 1211 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:59:36.776651 1226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:36.776814 1211 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:59:36.776914 1211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-1/wal
uuid: "2db81c82699747b3bd5bef1172282f3b"
format_stamp: "Formatted at 2026-09-29 10:59:36 on dist-test-slave-2lzq"
server_key: "c251c0a857b34b510e4a50922cdbc191"
server_key_iv: "7f98bbbc6a690a207761915e3b3e5e22"
server_key_version: "encryptionkey@0"
I20260929 10:59:36.776994 1211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:36.812141 1211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:36.814010 1211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:36.814203 1211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:36.818502 1211 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.253:33873
I20260929 10:59:36.818538 1278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.253:33873 every 8 connection(s)
I20260929 10:59:36.818961 1211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-1/data/info.pb
I20260929 10:59:36.819964 1279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:36.821483 1279 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:59:36.822328 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 1211
Sep 29 10:59:36 dist-test-slave-2lzq krb5kdc[702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679576, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.253@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM
Sep 29 10:59:36 dist-test-slave-2lzq krb5kdc[702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.120.253@KRBTEST.COM
I20260929 10:59:36.831437 786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:36.822202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60490 (local address 127.26.120.254:34785)
0929 10:59:36.822252 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:36.822254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:36.822568 (+ 314us) server_negotiation.cc:407] Connection header received
0929 10:59:36.823227 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:36.823229 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:36.823243 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:36.823298 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:36.823862 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.824372 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.825278 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.825474 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.825570 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:36.827948 (+ 2378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:36.827950 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:36.827950 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:36.827959 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:36.829786 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:36.830266 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:36.830267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:36.830268 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:36.830332 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:36.830641 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:36.830641 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:36.830642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:36.830855 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:36.830924 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:36.831237 (+ 313us) server_negotiation.cc:299] Negotiation successful
0929 10:59:36.831328 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 10:59:36.831594 1281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:36.822208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34785 (local address 127.0.0.1:60490)
0929 10:59:36.822418 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:36.822468 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:36.823128 (+ 660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:36.823336 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:36.823339 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:36.823602 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:36.823766 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:36.823771 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.824481 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.824482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:36.825184 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:36.825186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.825592 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.825593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:36.825714 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:36.826271 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:36.826281 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:36.827854 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:36.829869 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:36.829875 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:36.829877 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:36.830196 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:36.830404 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:36.830405 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:36.830405 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:36.830569 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:36.830925 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:36.830931 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:36.831105 (+ 174us) client_negotiation.cc:769] Sending connection context
0929 10:59:36.831186 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 10:59:36.831340 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":92,"threads_started":1}
I20260929 10:59:36.831686 1283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:36.822829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43386 (local address 127.26.120.253:33873)
0929 10:59:36.822971 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:36.822976 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:36.823037 (+ 61us) server_negotiation.cc:407] Connection header received
0929 10:59:36.823126 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:36.823139 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:36.823158 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:36.823255 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:36.823858 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.824673 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.825395 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.825706 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.825784 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:36.828525 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:36.828533 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:36.828533 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:36.828554 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:36.830412 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:36.830783 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:36.830786 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:36.830786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:36.830842 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:36.831063 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:36.831063 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:36.831063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:36.831220 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:36.831377 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:36.831453 (+ 76us) server_negotiation.cc:299] Negotiation successful
0929 10:59:36.831559 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":78,"threads_started":1}
I20260929 10:59:36.832358 1279 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 10:59:36 dist-test-slave-2lzq krb5kdc[702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679576, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.253@KRBTEST.COM for oryx/127.26.120.253@KRBTEST.COM
I20260929 10:59:36.837890 1281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:36.832590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.253:33873 (local address 127.0.0.1:43396)
0929 10:59:36.832671 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:36.832681 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:36.832722 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:36.832820 (+ 98us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:36.832821 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:36.832923 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:36.832965 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:36.832966 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.833526 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.833526 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:36.833976 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:36.833977 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.834356 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.834356 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:36.834409 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:36.834800 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:36.834809 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:36.835863 (+ 1054us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:36.837144 (+ 1281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:36.837145 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:36.837145 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:36.837290 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:36.837439 (+ 149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:36.837440 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:36.837440 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:36.837486 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:36.837664 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:36.837665 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:36.837696 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 10:59:36.837744 (+ 48us) client_negotiation.cc:240] Negotiation successful
0929 10:59:36.837813 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260929 10:59:36.837977 1283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:36.832639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43396 (local address 127.26.120.253:33873)
0929 10:59:36.832669 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:36.832671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:36.832712 (+ 41us) server_negotiation.cc:407] Connection header received
0929 10:59:36.832755 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:36.832756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:36.832764 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:36.832807 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:36.833017 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.833456 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.834042 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.834295 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.834347 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:36.835927 (+ 1580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:36.835928 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:36.835928 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:36.835933 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:36.837079 (+ 1146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:36.837350 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:36.837351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:36.837351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:36.837390 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:36.837533 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:36.837534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:36.837534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:36.837617 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:36.837659 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:36.837757 (+ 98us) server_negotiation.cc:299] Negotiation successful
0929 10:59:36.837853 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":13}
I20260929 10:59:36.839170 1279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b: Bootstrap starting.
I20260929 10:59:36.839918 1279 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:36.840142 1279 log.cc:824] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:36.840796 1279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b: No bootstrap required, opened a new log
I20260929 10:59:36.842228 1279 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } }
I20260929 10:59:36.842346 1279 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:36.842367 1279 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2db81c82699747b3bd5bef1172282f3b, State: Initialized, Role: FOLLOWER
I20260929 10:59:36.842485 1279 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } }
I20260929 10:59:36.842751 1279 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:59:36.842742 1285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } } }
I20260929 10:59:36.842872 1285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [sys.catalog]: This master's current role is: FOLLOWER
W20260929 10:59:36.844300 1296 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:59:36.844364 1296 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:59:36.845928 1211 master_runner.cc:428] Detected that this master 2db81c82699747b3bd5bef1172282f3b is joining an existing cluster
I20260929 10:59:36.845978 1211 master_runner.cc:432] Initiating AddMaster RPC to add 127.26.120.253:33873
I20260929 10:59:36.854204 1283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:36.847131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43400 (local address 127.26.120.253:33873)
0929 10:59:36.847211 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:36.847213 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:36.847514 (+ 301us) server_negotiation.cc:407] Connection header received
0929 10:59:36.847565 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:36.847566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:36.847575 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:36.847641 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:36.848109 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.848820 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.850126 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.850392 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.850453 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:36.851401 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:36.851402 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:36.851403 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:36.851409 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:36.853036 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:36.853392 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:36.853393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:36.853393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:36.853442 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:36.853647 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:36.853648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:36.853649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:36.853775 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:36.853834 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:36.853998 (+ 164us) server_negotiation.cc:299] Negotiation successful
0929 10:59:36.854120 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":58}
I20260929 10:59:36.854215 1303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:36.847030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.253:33873 (local address 127.0.0.1:43400)
0929 10:59:36.847434 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:36.847443 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:36.847517 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:36.847667 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:36.847668 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:36.847913 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:36.848013 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:36.848015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.848901 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.848902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:36.849687 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:36.849689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.850498 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.850499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:36.850537 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:36.850913 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:36.850917 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:36.851335 (+ 418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:36.853138 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:36.853138 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:36.853139 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:36.853307 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:36.853511 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:36.853512 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:36.853512 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:36.853568 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:36.853854 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:36.853855 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:36.853892 (+ 37us) client_negotiation.cc:769] Sending connection context
0929 10:59:36.853962 (+ 70us) client_negotiation.cc:240] Negotiation successful
0929 10:59:36.854089 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":115,"threads_started":1}
I20260929 10:59:36.854640 786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:36.847172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60502 (local address 127.26.120.254:34785)
0929 10:59:36.847214 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:36.847216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:36.847659 (+ 443us) server_negotiation.cc:407] Connection header received
0929 10:59:36.847737 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:36.847738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:36.847749 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:36.847803 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:36.848143 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.848651 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.849782 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.850123 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.850195 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:36.851345 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:36.851347 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:36.851347 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:36.851354 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:36.853323 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:36.853697 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:36.853698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:36.853699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:36.853751 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:36.853988 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:36.853989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:36.853990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:36.854153 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:36.854199 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:36.854425 (+ 226us) server_negotiation.cc:299] Negotiation successful
0929 10:59:36.854519 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 10:59:36.854970 1302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:36.847030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34785 (local address 127.0.0.1:60502)
0929 10:59:36.847590 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:36.847596 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:36.847682 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:36.847817 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:36.847818 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:36.847996 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:36.848049 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:36.848050 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.848753 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.848754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:36.849688 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:36.849689 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.850209 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.850209 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:36.850270 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:36.850768 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:36.850772 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:36.851259 (+ 487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:36.853415 (+ 2156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:36.853416 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:36.853416 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:36.853621 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:36.853822 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:36.853823 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:36.853823 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:36.853901 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:36.854249 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:36.854250 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:36.854299 (+ 49us) client_negotiation.cc:769] Sending connection context
0929 10:59:36.854765 (+ 466us) client_negotiation.cc:240] Negotiation successful
0929 10:59:36.854862 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"mutex_wait_us":5,"spinlock_wait_cycles":10624,"thread_start_us":123,"threads_started":1}
Sep 29 10:59:36 dist-test-slave-2lzq krb5kdc[702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679576, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.254@KRBTEST.COM for oryx/127.26.120.253@KRBTEST.COM
I20260929 10:59:36.864508 1283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:36.857045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43414 (local address 127.26.120.253:33873)
0929 10:59:36.857154 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:36.857155 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:36.857213 (+ 58us) server_negotiation.cc:407] Connection header received
0929 10:59:36.857318 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:36.857319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:36.857326 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:36.857367 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:36.857838 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.858356 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.859466 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.859642 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.859713 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:36.862102 (+ 2389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:36.862103 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:36.862103 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:36.862111 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:36.863409 (+ 1298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:36.863823 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:36.863824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:36.863824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:36.863870 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:36.864072 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:36.864072 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:36.864072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:36.864163 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:36.864211 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:36.864398 (+ 187us) server_negotiation.cc:299] Negotiation successful
0929 10:59:36.864449 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260929 10:59:36.864614 1304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:36.856974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.253:33873 (local address 127.0.0.1:43414)
0929 10:59:36.857118 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:36.857165 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:36.857238 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:36.857381 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:36.857384 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:36.857708 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:36.857767 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:36.857769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.858430 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.858431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:36.859379 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:36.859381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.859715 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.859716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:36.859818 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:36.860274 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:36.860280 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:36.861979 (+ 1699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:36.863487 (+ 1508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:36.863494 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:36.863496 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:36.863750 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:36.863933 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:36.863934 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:36.863934 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:36.863999 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:36.864226 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:36.864233 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:36.864289 (+ 56us) client_negotiation.cc:769] Sending connection context
0929 10:59:36.864383 (+ 94us) client_negotiation.cc:240] Negotiation successful
0929 10:59:36.864500 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":92,"threads_started":1}
I20260929 10:59:36.865221 747 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.26.120.253:33873: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "2db81c82699747b3bd5bef1172282f3b"
member_type: NON_VOTER
last_known_addr {
host: "127.26.120.253"
port: 33873
}
attrs {
promote: true
}
}
dest_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d"
cas_config_opid_index: -1
I20260929 10:59:36.865552 747 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: NON_VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: true } }
I20260929 10:59:36.866067 1306 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 2db81c82699747b3bd5bef1172282f3b (127.26.120.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: NON_VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: true } } }
I20260929 10:59:36.866376 1253 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:36.867031 1306 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.26.120.253:33873
I20260929 10:59:36.867254 1211 master.cc:537] Master@127.26.120.253:33873 shutting down...
I20260929 10:59:36.867326 1253 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [term 1 FOLLOWER]: Refusing update from remote peer 5c9a8ea9e35a43a99072b3ab64ca4b8d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260929 10:59:36.867406 1305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: NON_VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: true } } }
I20260929 10:59:36.867487 1305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:36.867635 1305 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: NON_VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 10:59:36.868343 731 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260929 10:59:36.868731 731 consensus_peers.cc:487] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d -> Peer 2db81c82699747b3bd5bef1172282f3b (127.26.120.253:33873): Couldn't send request to peer 2db81c82699747b3bd5bef1172282f3b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260929 10:59:36.869951 1211 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:59:36.870074 1211 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:36.870131 1211 tablet_replica.cc:330] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b: stopping tablet replica
I20260929 10:59:36.882675 1211 master.cc:552] Master@127.26.120.253:33873 shutdown complete.
I20260929 10:59:36.882727 1211 master_runner.cc:305] Clearing existing system tablet
I20260929 10:59:36.882856 1211 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:36.882905 1211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:36.883608 1211 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:59:36.884119 1310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:36.884259 1211 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:36.884318 1211 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:59:36.884341 1211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-1/wal
uuid: "2db81c82699747b3bd5bef1172282f3b"
format_stamp: "Formatted at 2026-09-29 10:59:36 on dist-test-slave-2lzq"
server_key: "c251c0a857b34b510e4a50922cdbc191"
server_key_iv: "7f98bbbc6a690a207761915e3b3e5e22"
server_key_version: "encryptionkey@0"
I20260929 10:59:36.885128 1211 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:36.886758 1211 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260929 10:59:36.886829 1211 log.cc:1196] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260929 10:59:36.887039 1211 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b: Deleting consensus metadata
I20260929 10:59:36.887157 1211 master_runner.cc:315] Copying system tablet from 127.26.120.254:34785
I20260929 10:59:36.887768 1211 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b: tablet copy: Beginning tablet copy session from remote peer at address 127.26.120.254:34785
I20260929 10:59:36.894012 786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:36.888118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60514 (local address 127.26.120.254:34785)
0929 10:59:36.888168 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:36.888170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:36.888285 (+ 115us) server_negotiation.cc:407] Connection header received
0929 10:59:36.888345 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:36.888346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:36.888355 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:36.888399 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:36.888735 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.889228 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.890386 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.890573 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.890628 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:36.891671 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:36.891672 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:36.891672 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:36.891678 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:36.892985 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:36.893349 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:36.893350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:36.893350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:36.893398 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:36.893589 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:36.893589 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:36.893590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:36.893684 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:36.893734 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:36.893894 (+ 160us) server_negotiation.cc:299] Negotiation successful
0929 10:59:36.893951 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 10:59:36.894098 1316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:36.888024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34785 (local address 127.0.0.1:60514)
0929 10:59:36.888207 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:36.888217 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:36.888282 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:36.888413 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:36.888414 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:36.888603 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:36.888669 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:36.888671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.889329 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.889331 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:36.890302 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:36.890305 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.890650 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.890651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:36.890723 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:36.891164 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:36.891169 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:36.891585 (+ 416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:36.893063 (+ 1478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:36.893064 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:36.893065 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:36.893279 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:36.893463 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:36.893464 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:36.893465 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:36.893529 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:36.893744 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:36.893745 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:36.893790 (+ 45us) client_negotiation.cc:769] Sending connection context
0929 10:59:36.893868 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 10:59:36.893967 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":96,"threads_started":1}
I20260929 10:59:36.894650 767 tablet_copy_service.cc:140] P 5c9a8ea9e35a43a99072b3ab64ca4b8d: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 2db81c82699747b3bd5bef1172282f3b ({username='oryx', principal='oryx/127.26.120.253@KRBTEST.COM'} at 127.0.0.1:60514)
I20260929 10:59:36.894843 767 tablet_copy_service.cc:161] P 5c9a8ea9e35a43a99072b3ab64ca4b8d: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 2db81c82699747b3bd5bef1172282f3b at {username='oryx', principal='oryx/127.26.120.253@KRBTEST.COM'} at 127.0.0.1:60514: session id = 2db81c82699747b3bd5bef1172282f3b-00000000000000000000000000000000
I20260929 10:59:36.895720 767 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d: Tablet Copy: opened 0 blocks and 1 log segments
I20260929 10:59:36.896311 1211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260929 10:59:36.897217 1211 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b: tablet copy: Starting download of 0 data blocks...
I20260929 10:59:36.897387 1211 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b: tablet copy: Starting download of 1 WAL segments...
I20260929 10:59:36.898264 1211 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260929 10:59:36.899031 767 tablet_copy_service.cc:342] P 5c9a8ea9e35a43a99072b3ab64ca4b8d: Request end of tablet copy session 2db81c82699747b3bd5bef1172282f3b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.26.120.253@KRBTEST.COM'} at 127.0.0.1:60514
I20260929 10:59:36.899099 767 tablet_copy_service.cc:433] P 5c9a8ea9e35a43a99072b3ab64ca4b8d: ending tablet copy session 2db81c82699747b3bd5bef1172282f3b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 2db81c82699747b3bd5bef1172282f3b
I20260929 10:59:36.901315 1211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:36.902554 1317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:36.902616 1318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:36.902630 1320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:36.902781 1211 server_base.cc:1034] running on GCE node
I20260929 10:59:36.902869 1211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:36.903029 1211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:36.904165 1211 hybrid_clock.cc:648] HybridClock initialized: now 1790679576904157 us; error 24 us; skew 500 ppm
Sep 29 10:59:36 dist-test-slave-2lzq krb5kdc[702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679576, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:36.905457 1211 init.cc:377] Logged in from keytab as oryx/127.26.120.253@KRBTEST.COM (short username oryx)
I20260929 10:59:36.905742 1211 webserver.cc:460] Webserver started at http://127.26.120.253:43967/ using document root <none> and password file <none>
I20260929 10:59:36.905871 1211 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:36.905910 1211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:36.906589 1211 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:36.907105 1327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:36.907222 1211 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:59:36.907269 1211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-1/wal
uuid: "2db81c82699747b3bd5bef1172282f3b"
format_stamp: "Formatted at 2026-09-29 10:59:36 on dist-test-slave-2lzq"
server_key: "c251c0a857b34b510e4a50922cdbc191"
server_key_iv: "7f98bbbc6a690a207761915e3b3e5e22"
server_key_version: "encryptionkey@0"
I20260929 10:59:36.907323 1211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:36.915839 1211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:36.917659 1211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:36.920815 1211 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.253:33873
I20260929 10:59:36.921144 1379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.253:33873 every 8 connection(s)
I20260929 10:59:36.921249 1211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-1/data/info.pb
I20260929 10:59:36.922190 1380 sys_catalog.cc:263] Verifying existing consensus state
I20260929 10:59:36.922721 1380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b: Bootstrap starting.
I20260929 10:59:36.925767 1380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 10:59:36.926039 1380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b: Bootstrap complete.
I20260929 10:59:36.926263 1380 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: NON_VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: true } }
I20260929 10:59:36.926452 1380 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2db81c82699747b3bd5bef1172282f3b, State: Initialized, Role: LEARNER
I20260929 10:59:36.926503 1380 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: NON_VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: true } }
I20260929 10:59:36.926713 1380 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:59:36.926699 1383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: NON_VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: true } } }
I20260929 10:59:36.926820 1383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [sys.catalog]: This master's current role is: LEARNER
I20260929 10:59:36.928417 1394 catalog_manager.cc:1259] Loaded cluster ID: 27ec5d70879346ffba3bed1fd19d6dff
I20260929 10:59:36.928499 1394 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b: loading cluster ID for follower catalog manager: success
I20260929 10:59:36.929605 1394 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b: acquiring CA information for follower catalog manager: success
I20260929 10:59:36.929880 1394 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 10:59:36.940388 1395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:36.933226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43436 (local address 127.26.120.253:33873)
0929 10:59:36.933362 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:36.933365 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:36.933375 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:59:36.933401 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:36.933402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:36.933413 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:36.933510 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:36.934027 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.935184 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.935882 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:36.936214 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:36.936301 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:36.937501 (+ 1200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:36.937503 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:36.937503 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:36.937511 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:36.939245 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:36.939574 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:36.939576 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:36.939576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:36.939645 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:36.939837 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:36.939838 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:36.939838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:36.939975 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:36.940089 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:36.940169 (+ 80us) server_negotiation.cc:299] Negotiation successful
0929 10:59:36.940254 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":57,"threads_started":1}
WARNING: no policy specified for oryx/127.26.120.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.120.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.120.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.252.keytab.
Entry for principal oryx/127.26.120.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.252.keytab.
I20260929 10:59:36.964928 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.252:36809
--webserver_interface=127.26.120.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.252.keytab
--principal=oryx/127.26.120.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:41445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:34785,127.26.120.253:33873,127.26.120.252:36809
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:37.048063 1398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:37.048254 1398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:37.048293 1398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:37.049368 1398 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:59:37.049403 1398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:37.049418 1398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:37.049430 1398 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:59:37.049443 1398 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:59:37.050945 1398 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:41445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:34785,127.26.120.253:33873,127.26.120.252:36809
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.252.keytab
--principal=oryx/127.26.120.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.252:36809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.120.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:37.051151 1398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:37.051322 1398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 10:59:37.056106 1398 server_base.cc:1034] running on GCE node
W20260929 10:59:37.056049 1406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:37.056049 1404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:37.056058 1403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:37.056504 1398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:37.056829 1398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:37.057996 1398 hybrid_clock.cc:648] HybridClock initialized: now 1790679577057982 us; error 42 us; skew 500 ppm
Sep 29 10:59:37 dist-test-slave-2lzq krb5kdc[702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679577, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:37.064121 1398 init.cc:377] Logged in from keytab as oryx/127.26.120.252@KRBTEST.COM (short username oryx)
I20260929 10:59:37.064551 1398 webserver.cc:460] Webserver started at http://127.26.120.252:36589/ using document root <none> and password file <none>
I20260929 10:59:37.064785 1398 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:37.064852 1398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:37.065003 1398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:37.066275 1398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-2/data/instance:
uuid: "7e0f409104114e16aa509d93ea23e480"
format_stamp: "Formatted at 2026-09-29 10:59:37 on dist-test-slave-2lzq"
server_key: "74a710c93fbf874315a8054eec825d10"
server_key_iv: "660adf197a7e154cdf03ad17ae3e1df0"
server_key_version: "encryptionkey@0"
I20260929 10:59:37.066692 1398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-2/wal/instance:
uuid: "7e0f409104114e16aa509d93ea23e480"
format_stamp: "Formatted at 2026-09-29 10:59:37 on dist-test-slave-2lzq"
server_key: "74a710c93fbf874315a8054eec825d10"
server_key_iv: "660adf197a7e154cdf03ad17ae3e1df0"
server_key_version: "encryptionkey@0"
I20260929 10:59:37.068081 1398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 10:59:37.068856 1413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:37.069072 1398 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:59:37.069172 1398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-2/wal
uuid: "7e0f409104114e16aa509d93ea23e480"
format_stamp: "Formatted at 2026-09-29 10:59:37 on dist-test-slave-2lzq"
server_key: "74a710c93fbf874315a8054eec825d10"
server_key_iv: "660adf197a7e154cdf03ad17ae3e1df0"
server_key_version: "encryptionkey@0"
I20260929 10:59:37.069257 1398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:37.088908 1398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:37.091416 1398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:37.091590 1398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:37.095579 1398 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.252:36809
I20260929 10:59:37.095623 1465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.252:36809 every 8 connection(s)
I20260929 10:59:37.095997 1398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-2/data/info.pb
I20260929 10:59:37.096931 1466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:37.098416 1466 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:59:37.100098 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 1398
Sep 29 10:59:37 dist-test-slave-2lzq krb5kdc[702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679577, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.252@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM
Sep 29 10:59:37 dist-test-slave-2lzq krb5kdc[702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.120.252@KRBTEST.COM
I20260929 10:59:37.109671 786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.099250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60516 (local address 127.26.120.254:34785)
0929 10:59:37.099296 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:37.099298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:37.099339 (+ 41us) server_negotiation.cc:407] Connection header received
0929 10:59:37.099794 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:37.099795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:37.099803 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:37.099869 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:37.100766 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.101616 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.103024 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.103316 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.103397 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.106041 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:37.106042 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:37.106043 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:37.106072 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:37.107916 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.108566 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.108567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.108568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.108628 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.108924 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.108925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.108926 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.109081 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:37.109149 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:37.109483 (+ 334us) server_negotiation.cc:299] Negotiation successful
0929 10:59:37.109571 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 10:59:37.109884 1468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.099059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34785 (local address 127.0.0.1:60516)
0929 10:59:37.099226 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:37.099269 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:37.099711 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:37.099978 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:37.099983 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:37.100415 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:37.100651 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.100657 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.101710 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.101711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.102904 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.102907 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.103418 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.103419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.103525 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.104046 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:37.104056 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:37.105907 (+ 1851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:37.108002 (+ 2095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:37.108009 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:37.108011 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:37.108486 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:37.108697 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:37.108698 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:37.108698 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:37.108826 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:37.109152 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:37.109159 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:37.109322 (+ 163us) client_negotiation.cc:769] Sending connection context
0929 10:59:37.109450 (+ 128us) client_negotiation.cc:240] Negotiation successful
0929 10:59:37.109618 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":97,"threads_started":1}
I20260929 10:59:37.110525 1466 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } has no permanent_uuid. Determining permanent_uuid...
I20260929 10:59:37.110661 1469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.100649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48240 (local address 127.26.120.252:36809)
0929 10:59:37.100836 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:37.100846 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:37.100859 (+ 13us) server_negotiation.cc:407] Connection header received
0929 10:59:37.100876 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:37.100877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:37.100895 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:37.100953 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:37.101710 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.102946 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.103810 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.104261 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.104333 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.107123 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:37.107130 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:37.107130 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:37.107136 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:37.109056 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.109466 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.109469 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.109469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.109549 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.109779 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.109780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.109780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.109962 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:37.110502 (+ 540us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:37.110539 (+ 37us) server_negotiation.cc:299] Negotiation successful
0929 10:59:37.110596 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":165,"thread_start_us":99,"threads_started":1}
Sep 29 10:59:37 dist-test-slave-2lzq krb5kdc[702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679577, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.252@KRBTEST.COM for oryx/127.26.120.253@KRBTEST.COM
I20260929 10:59:37.116789 1468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.110839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.253:33873 (local address 127.0.0.1:43450)
0929 10:59:37.110873 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:37.110884 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:37.110926 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:37.111072 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:37.111073 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:37.111212 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:37.111264 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.111267 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.111964 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.111965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.112451 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.112452 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.112878 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.112879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.112946 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.113274 (+ 328us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:37.113278 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:37.114450 (+ 1172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:37.115873 (+ 1423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:37.115873 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:37.115874 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:37.116107 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:37.116307 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:37.116307 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:37.116308 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:37.116358 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:37.116577 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:37.116577 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:37.116612 (+ 35us) client_negotiation.cc:769] Sending connection context
0929 10:59:37.116663 (+ 51us) client_negotiation.cc:240] Negotiation successful
0929 10:59:37.116729 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260929 10:59:37.116822 1395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.110886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43450 (local address 127.26.120.253:33873)
0929 10:59:37.110946 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:37.110948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:37.110958 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:59:37.110991 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:37.110992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:37.111003 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:37.111048 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:37.111379 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.111895 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.112577 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.112794 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.112879 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.114523 (+ 1644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:37.114524 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:37.114524 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:37.114531 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:37.115802 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.116200 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.116201 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.116201 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.116250 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.116430 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.116430 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.116430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.116521 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:37.116572 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:37.116683 (+ 111us) server_negotiation.cc:299] Negotiation successful
0929 10:59:37.116747 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 10:59:37.117220 1466 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.120.252" port: 36809 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 10:59:37 dist-test-slave-2lzq krb5kdc[702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679577, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.252@KRBTEST.COM for oryx/127.26.120.252@KRBTEST.COM
I20260929 10:59:37.123368 1468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.117554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:36809 (local address 127.0.0.1:48244)
0929 10:59:37.117607 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:37.117616 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:37.117657 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:37.117778 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:37.117779 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:37.117879 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:37.117928 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.117929 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.118729 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.118730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.119183 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.119185 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.119508 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.119509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.119548 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.119952 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:37.119956 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:37.121019 (+ 1063us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:37.122553 (+ 1534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:37.122553 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:37.122554 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:37.122703 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:37.122886 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:37.122886 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:37.122886 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:37.122938 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:37.123132 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:37.123132 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:37.123166 (+ 34us) client_negotiation.cc:769] Sending connection context
0929 10:59:37.123216 (+ 50us) client_negotiation.cc:240] Negotiation successful
0929 10:59:37.123287 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260929 10:59:37.123416 1469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.117623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48244 (local address 127.26.120.252:36809)
0929 10:59:37.117657 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:37.117659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:37.117693 (+ 34us) server_negotiation.cc:407] Connection header received
0929 10:59:37.117712 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:37.117713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:37.117721 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:37.117774 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:37.117992 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.118630 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.119246 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.119444 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.119496 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.121091 (+ 1595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:37.121092 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:37.121092 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:37.121097 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:37.122482 (+ 1385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.122776 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.122777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.122777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.122828 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.122988 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.122989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.122989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.123082 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:37.123127 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:37.123229 (+ 102us) server_negotiation.cc:299] Negotiation successful
0929 10:59:37.123328 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":15}
I20260929 10:59:37.124801 1466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480: Bootstrap starting.
I20260929 10:59:37.125583 1466 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:37.125813 1466 log.cc:824] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:37.126524 1466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480: No bootstrap required, opened a new log
I20260929 10:59:37.128099 1466 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } } peers { permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 36809 } }
I20260929 10:59:37.128221 1466 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:37.128243 1466 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e0f409104114e16aa509d93ea23e480, State: Initialized, Role: FOLLOWER
I20260929 10:59:37.128392 1466 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } } peers { permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 36809 } }
I20260929 10:59:37.128695 1466 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:59:37.128660 1471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } } peers { permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 36809 } } }
I20260929 10:59:37.128759 1471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [sys.catalog]: This master's current role is: FOLLOWER
W20260929 10:59:37.130380 1482 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 10:59:37.130442 1482 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 10:59:37.132732 1398 master_runner.cc:428] Detected that this master 7e0f409104114e16aa509d93ea23e480 is joining an existing cluster
I20260929 10:59:37.132781 1398 master_runner.cc:432] Initiating AddMaster RPC to add 127.26.120.252:36809
I20260929 10:59:37.158223 786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.133782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60526 (local address 127.26.120.254:34785)
0929 10:59:37.133824 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:37.133826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:37.134374 (+ 548us) server_negotiation.cc:407] Connection header received
0929 10:59:37.134420 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:37.134420 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:37.134432 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:37.134488 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:37.135791 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.136507 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.137474 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.137766 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.137838 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.155098 (+ 17260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:37.155100 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:37.155101 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:37.155109 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:37.156842 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.157292 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.157293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.157293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.157369 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.157566 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.157567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.157567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.157706 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:37.157757 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:37.157998 (+ 241us) server_negotiation.cc:299] Negotiation successful
0929 10:59:37.158117 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 10:59:37.158291 1469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.134092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48248 (local address 127.26.120.252:36809)
0929 10:59:37.134131 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:37.134133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:37.134659 (+ 526us) server_negotiation.cc:407] Connection header received
0929 10:59:37.134679 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:37.134680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:37.134689 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:37.134736 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:37.135046 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.135772 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.136799 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.137071 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.137120 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.155292 (+ 18172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:37.155293 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:37.155294 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:37.155299 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:37.157085 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.157479 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.157480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.157480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.157528 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.157721 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.157722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.157722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.157850 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:37.157893 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:37.158099 (+ 206us) server_negotiation.cc:299] Negotiation successful
0929 10:59:37.158175 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 10:59:37.158629 1489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.134168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:36809 (local address 127.0.0.1:48248)
0929 10:59:37.134570 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:37.134579 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:37.134624 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:37.134737 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:37.134738 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:37.134901 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:37.134963 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.134964 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.135859 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.135860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.136708 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.136711 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.137186 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.137186 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.137249 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.137671 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:37.137675 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:37.155081 (+ 17406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:37.157170 (+ 2089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:37.157170 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:37.157171 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:37.157401 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:37.157607 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:37.157607 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:37.157607 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:37.157658 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:37.157933 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:37.157933 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:37.157968 (+ 35us) client_negotiation.cc:769] Sending connection context
0929 10:59:37.158405 (+ 437us) client_negotiation.cc:240] Negotiation successful
0929 10:59:37.158501 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":62,"threads_started":1}
I20260929 10:59:37.158895 1488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.133773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.253:33873 (local address 127.0.0.1:43462)
0929 10:59:37.133910 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:37.133923 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:37.133976 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:37.134133 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:37.134133 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:37.134379 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:37.134467 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.134469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.135335 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.135335 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.136230 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.136232 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.136730 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.136731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.136790 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.137277 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:37.137282 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:37.155081 (+ 17799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:37.157806 (+ 2725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:37.157807 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:37.157807 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:37.158008 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:37.158238 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:37.158238 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:37.158239 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:37.158312 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:37.158597 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:37.158598 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:37.158634 (+ 36us) client_negotiation.cc:769] Sending connection context
0929 10:59:37.158698 (+ 64us) client_negotiation.cc:240] Negotiation successful
0929 10:59:37.158802 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":86,"threads_started":1}
I20260929 10:59:37.159245 1395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.133934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43462 (local address 127.26.120.253:33873)
0929 10:59:37.133975 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:37.133977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:37.133986 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:59:37.134034 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:37.134035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:37.134045 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:37.134114 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:37.134531 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.135256 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.136314 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.136612 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.136669 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.155262 (+ 18593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:37.155264 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:37.155264 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:37.155272 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:37.157726 (+ 2454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.158117 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.158118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.158119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.158172 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.158380 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.158381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.158381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.158512 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:37.158572 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:37.159078 (+ 506us) server_negotiation.cc:299] Negotiation successful
0929 10:59:37.159156 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 10:59:37.159373 1490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.133864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34785 (local address 127.0.0.1:60526)
0929 10:59:37.134289 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:37.134300 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:37.134356 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:37.135424 (+ 1068us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:37.135425 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:37.135634 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:37.135696 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.135697 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.136603 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.136604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.137394 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.137397 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.137837 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.137837 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.137884 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.138274 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:37.138279 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:37.154953 (+ 16674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:37.156950 (+ 1997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:37.156951 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:37.156951 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:37.157187 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:37.157442 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:37.157443 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:37.157443 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:37.157498 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:37.157818 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:37.157819 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:37.157869 (+ 50us) client_negotiation.cc:769] Sending connection context
0929 10:59:37.158895 (+ 1026us) client_negotiation.cc:240] Negotiation successful
0929 10:59:37.159269 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":68,"threads_started":1}
I20260929 10:59:37.162593 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 809
Sep 29 10:59:37 dist-test-slave-2lzq krb5kdc[702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679576, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.254@KRBTEST.COM for oryx/127.26.120.252@KRBTEST.COM
I20260929 10:59:37.167876 1469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.161243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48260 (local address 127.26.120.252:36809)
0929 10:59:37.161276 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:37.161278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:37.161285 (+ 7us) server_negotiation.cc:407] Connection header received
0929 10:59:37.161316 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:37.161316 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:37.161325 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:37.161372 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:37.161662 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.162150 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.163071 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.163342 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.163431 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.165276 (+ 1845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:37.165278 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:37.165278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:37.165285 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:37.166707 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.167145 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.167146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.167146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.167200 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.167388 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.167389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.167389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.167493 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:37.167580 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:37.167724 (+ 144us) server_negotiation.cc:299] Negotiation successful
0929 10:59:37.167793 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260929 10:59:37.167882 1304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.161175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:36809 (local address 127.0.0.1:48260)
0929 10:59:37.161220 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:37.161230 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:37.161264 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:37.161384 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:37.161385 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:37.161542 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:37.161587 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.161589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.162215 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.162216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.162977 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.162979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.163432 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.163433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.163501 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.163910 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:37.163915 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:37.165187 (+ 1272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:37.166814 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:37.166815 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:37.166816 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:37.167061 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:37.167265 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:37.167266 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:37.167266 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:37.167327 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:37.167575 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:37.167576 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:37.167619 (+ 43us) client_negotiation.cc:769] Sending connection context
0929 10:59:37.167702 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 10:59:37.167794 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260929 10:59:37.168694 747 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.26.120.252:36809: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "7e0f409104114e16aa509d93ea23e480"
member_type: NON_VOTER
last_known_addr {
host: "127.26.120.252"
port: 36809
}
attrs {
promote: true
}
}
dest_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d"
cas_config_opid_index: 5
I20260929 10:59:37.168947 747 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: NON_VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: true } } peers { permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: NON_VOTER last_known_addr { host: "127.26.120.252" port: 36809 } attrs { promote: true } }
I20260929 10:59:37.169307 1306 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 7e0f409104114e16aa509d93ea23e480 (127.26.120.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: NON_VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: true } } peers { permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: NON_VOTER last_known_addr { host: "127.26.120.252" port: 36809 } attrs { promote: true } } }
I20260929 10:59:37.169533 1440 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:37.170292 1306 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.26.120.252:36809
I20260929 10:59:37.170369 1306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: NON_VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: true } } peers { permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: NON_VOTER last_known_addr { host: "127.26.120.252" port: 36809 } attrs { promote: true } } }
I20260929 10:59:37.170439 1306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:37.170480 1440 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [term 1 FOLLOWER]: Refusing update from remote peer 5c9a8ea9e35a43a99072b3ab64ca4b8d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260929 10:59:37.170724 1305 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: NON_VOTER last_known_addr { host: "127.26.120.252" port: 36809 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:59:37.170960 1398 master.cc:537] Master@127.26.120.252:36809 shutting down...
W20260929 10:59:37.171540 732 consensus_peers.cc:487] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d -> Peer 7e0f409104114e16aa509d93ea23e480 (127.26.120.252:36809): Couldn't send request to peer 7e0f409104114e16aa509d93ea23e480. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260929 10:59:37.171867 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.193:42371
--local_ip_for_outbound_sockets=127.26.120.193
--tserver_master_addrs=127.26.120.254:34785
--webserver_port=41851
--webserver_interface=127.26.120.193
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.193.keytab
--principal=oryx/127.26.120.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:41445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.26.120.254:34785,127.26.120.253:33873
--tserver_master_addrs=127.26.120.254:34785,127.26.120.253:33873,127.26.120.252:36809 with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:59:37.173947 1398 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:59:37.174072 1398 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:37.174129 1398 tablet_replica.cc:330] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480: stopping tablet replica
I20260929 10:59:37.176191 1304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.169875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.253:33873 (local address 127.0.0.1:43476)
0929 10:59:37.170514 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:37.170524 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:37.170562 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:37.170780 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:37.170781 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:37.170782 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:37.170823 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.170824 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.173496 (+ 2672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.173497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.174872 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.174876 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.175901 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.175902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.175973 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.176019 (+ 46us) client_negotiation.cc:769] Sending connection context
0929 10:59:37.176080 (+ 61us) client_negotiation.cc:240] Negotiation successful
0929 10:59:37.176122 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626}
I20260929 10:59:37.176452 1395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.169971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43476 (local address 127.26.120.253:33873)
0929 10:59:37.170006 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:37.170008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:37.170612 (+ 604us) server_negotiation.cc:407] Connection header received
0929 10:59:37.170635 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:37.170636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:37.170658 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:37.170710 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:37.170887 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.173402 (+ 2515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.174971 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.175570 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.175662 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.175864 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:37.176349 (+ 485us) server_negotiation.cc:299] Negotiation successful
0929 10:59:37.176391 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 10:59:37.176709 1398 master.cc:552] Master@127.26.120.252:36809 shutdown complete.
I20260929 10:59:37.176709 1354 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [term 1 LEARNER]: Refusing update from remote peer 5c9a8ea9e35a43a99072b3ab64ca4b8d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260929 10:59:37.176757 1398 master_runner.cc:305] Clearing existing system tablet
I20260929 10:59:37.176883 1398 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:37.176927 1398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:37.176924 1305 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: NON_VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260929 10:59:37.177507 1354 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 7e0f409104114e16aa509d93ea23e480 (127.26.120.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: NON_VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: true } } peers { permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: NON_VOTER last_known_addr { host: "127.26.120.252" port: 36809 } attrs { promote: true } } }
I20260929 10:59:37.178675 1398 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260929 10:59:37.178730 1383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c9a8ea9e35a43a99072b3ab64ca4b8d. Latest consensus state: current_term: 1 leader_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: NON_VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: true } } peers { permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: NON_VOTER last_known_addr { host: "127.26.120.252" port: 36809 } attrs { promote: true } } }
I20260929 10:59:37.178828 1383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [sys.catalog]: This master's current role is: LEARNER
I20260929 10:59:37.178871 1383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: NON_VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: true } } peers { permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: NON_VOTER last_known_addr { host: "127.26.120.252" port: 36809 } attrs { promote: true } } }
I20260929 10:59:37.178928 1383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [sys.catalog]: This master's current role is: LEARNER
I20260929 10:59:37.179193 1495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:37.179286 1305 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d: attempting to promote NON_VOTER 2db81c82699747b3bd5bef1172282f3b to VOTER
I20260929 10:59:37.179289 1398 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:37.179329 1398 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:37.179349 1398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-2/wal
uuid: "7e0f409104114e16aa509d93ea23e480"
format_stamp: "Formatted at 2026-09-29 10:59:37 on dist-test-slave-2lzq"
server_key: "74a710c93fbf874315a8054eec825d10"
server_key_iv: "660adf197a7e154cdf03ad17ae3e1df0"
server_key_version: "encryptionkey@0"
I20260929 10:59:37.179483 1305 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: false } } peers { permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: NON_VOTER last_known_addr { host: "127.26.120.252" port: 36809 } attrs { promote: true } }
I20260929 10:59:37.180171 1398 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:37.180310 1354 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [term 1 LEARNER]: Refusing update from remote peer 5c9a8ea9e35a43a99072b3ab64ca4b8d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260929 10:59:37.180497 1306 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260929 10:59:37.181172 1306 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 2db81c82699747b3bd5bef1172282f3b (127.26.120.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: false } } peers { permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: NON_VOTER last_known_addr { host: "127.26.120.252" port: 36809 } attrs { promote: true } } }
I20260929 10:59:37.181842 1304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.181727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:36809 (local address 127.0.0.1:48270)
0929 10:59:37.181770 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:37.181808 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.252:36809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260929 10:59:37.182157 1304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.182074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:36809 (local address 127.0.0.1:48274)
0929 10:59:37.182110 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:37.182133 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.252:36809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260929 10:59:37.182152 1306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: false } } peers { permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: NON_VOTER last_known_addr { host: "127.26.120.252" port: 36809 } attrs { promote: true } } }
W20260929 10:59:37.182230 732 consensus_peers.cc:487] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d -> Peer 7e0f409104114e16aa509d93ea23e480 (127.26.120.252:36809): Couldn't send request to peer 7e0f409104114e16aa509d93ea23e480. Status: Network error: Client connection negotiation failed: client connection to 127.26.120.252:36809: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260929 10:59:37.182236 1306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:37.182327 1306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: false } } peers { permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: NON_VOTER last_known_addr { host: "127.26.120.252" port: 36809 } attrs { promote: true } } }
I20260929 10:59:37.182385 1306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:37.182559 1354 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 2db81c82699747b3bd5bef1172282f3b (127.26.120.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: false } } peers { permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: NON_VOTER last_known_addr { host: "127.26.120.252" port: 36809 } attrs { promote: true } } }
I20260929 10:59:37.183233 1383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: false } } peers { permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: NON_VOTER last_known_addr { host: "127.26.120.252" port: 36809 } attrs { promote: true } } }
I20260929 10:59:37.183310 1383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:59:37.183523 1398 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260929 10:59:37.183634 1398 log.cc:1196] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260929 10:59:37.183835 1398 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480: Deleting consensus metadata
I20260929 10:59:37.183972 1398 master_runner.cc:315] Copying system tablet from 127.26.120.254:34785
I20260929 10:59:37.184590 1398 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480: tablet copy: Beginning tablet copy session from remote peer at address 127.26.120.254:34785
I20260929 10:59:37.192459 786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.185031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60532 (local address 127.26.120.254:34785)
0929 10:59:37.185078 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:37.185080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:37.185087 (+ 7us) server_negotiation.cc:407] Connection header received
0929 10:59:37.185105 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:37.185106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:37.185115 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:37.185162 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:37.185476 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.185938 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.186722 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.186927 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.187011 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.188081 (+ 1070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:37.188082 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:37.188083 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:37.188090 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:37.191296 (+ 3206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.191670 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.191671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.191672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.191736 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.191937 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.191938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.191938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.192060 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:37.192134 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:37.192297 (+ 163us) server_negotiation.cc:299] Negotiation successful
0929 10:59:37.192365 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260929 10:59:37.192490 1502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.184859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34785 (local address 127.0.0.1:60532)
0929 10:59:37.184974 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:37.184985 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:37.185035 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:37.185166 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:37.185167 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:37.185349 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:37.185388 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.185390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.186028 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.186029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.186631 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.186633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.187014 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.187015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.187073 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.187568 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:37.187575 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:37.187975 (+ 400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:37.191400 (+ 3425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:37.191401 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:37.191401 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:37.191580 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:37.191813 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:37.191814 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:37.191814 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:37.191870 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:37.192138 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:37.192139 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:37.192187 (+ 48us) client_negotiation.cc:769] Sending connection context
0929 10:59:37.192257 (+ 70us) client_negotiation.cc:240] Negotiation successful
0929 10:59:37.192365 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":82,"threads_started":1}
I20260929 10:59:37.193063 767 tablet_copy_service.cc:140] P 5c9a8ea9e35a43a99072b3ab64ca4b8d: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 7e0f409104114e16aa509d93ea23e480 ({username='oryx', principal='oryx/127.26.120.252@KRBTEST.COM'} at 127.0.0.1:60532)
I20260929 10:59:37.193136 767 tablet_copy_service.cc:161] P 5c9a8ea9e35a43a99072b3ab64ca4b8d: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 7e0f409104114e16aa509d93ea23e480 at {username='oryx', principal='oryx/127.26.120.252@KRBTEST.COM'} at 127.0.0.1:60532: session id = 7e0f409104114e16aa509d93ea23e480-00000000000000000000000000000000
I20260929 10:59:37.193571 767 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d: Tablet Copy: opened 0 blocks and 1 log segments
I20260929 10:59:37.194114 1398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260929 10:59:37.195077 1398 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480: tablet copy: Starting download of 0 data blocks...
I20260929 10:59:37.195238 1398 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480: tablet copy: Starting download of 1 WAL segments...
I20260929 10:59:37.195917 1398 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260929 10:59:37.196635 767 tablet_copy_service.cc:342] P 5c9a8ea9e35a43a99072b3ab64ca4b8d: Request end of tablet copy session 7e0f409104114e16aa509d93ea23e480-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.26.120.252@KRBTEST.COM'} at 127.0.0.1:60532
I20260929 10:59:37.196684 767 tablet_copy_service.cc:433] P 5c9a8ea9e35a43a99072b3ab64ca4b8d: ending tablet copy session 7e0f409104114e16aa509d93ea23e480-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 7e0f409104114e16aa509d93ea23e480
I20260929 10:59:37.199354 1398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:37.200577 1503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:37.200760 1504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:37.200901 1398 server_base.cc:1034] running on GCE node
W20260929 10:59:37.202181 1506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:37.202342 1398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:37.202494 1398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:37.204999 1398 hybrid_clock.cc:648] HybridClock initialized: now 1790679577204989 us; error 27 us; skew 500 ppm
Sep 29 10:59:37 dist-test-slave-2lzq krb5kdc[702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679577, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:37.207146 1398 init.cc:377] Logged in from keytab as oryx/127.26.120.252@KRBTEST.COM (short username oryx)
I20260929 10:59:37.207635 1398 webserver.cc:460] Webserver started at http://127.26.120.252:45431/ using document root <none> and password file <none>
I20260929 10:59:37.207762 1398 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:37.207796 1398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:37.216634 1398 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.001s sys 0.000s
I20260929 10:59:37.217543 1513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:37.217691 1398 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:37.217741 1398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-2/wal
uuid: "7e0f409104114e16aa509d93ea23e480"
format_stamp: "Formatted at 2026-09-29 10:59:37 on dist-test-slave-2lzq"
server_key: "74a710c93fbf874315a8054eec825d10"
server_key_iv: "660adf197a7e154cdf03ad17ae3e1df0"
server_key_version: "encryptionkey@0"
I20260929 10:59:37.217813 1398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:37.235998 1398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:37.237972 1398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:37.241206 1398 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.252:36809
I20260929 10:59:37.241222 1565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.252:36809 every 8 connection(s)
I20260929 10:59:37.241551 1398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/master-2/data/info.pb
I20260929 10:59:37.246963 1566 sys_catalog.cc:263] Verifying existing consensus state
I20260929 10:59:37.247655 1566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480: Bootstrap starting.
I20260929 10:59:37.251334 1566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 10:59:37.251590 1566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480: Bootstrap complete.
I20260929 10:59:37.251807 1566 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: false } } peers { permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: NON_VOTER last_known_addr { host: "127.26.120.252" port: 36809 } attrs { promote: true } }
I20260929 10:59:37.251916 1566 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7e0f409104114e16aa509d93ea23e480, State: Initialized, Role: LEARNER
I20260929 10:59:37.251960 1566 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: false } } peers { permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: NON_VOTER last_known_addr { host: "127.26.120.252" port: 36809 } attrs { promote: true } }
I20260929 10:59:37.252158 1566 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:59:37.252140 1569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: false } } peers { permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: NON_VOTER last_known_addr { host: "127.26.120.252" port: 36809 } attrs { promote: true } } }
I20260929 10:59:37.252359 1569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [sys.catalog]: This master's current role is: LEARNER
I20260929 10:59:37.254177 1580 catalog_manager.cc:1259] Loaded cluster ID: 27ec5d70879346ffba3bed1fd19d6dff
I20260929 10:59:37.254262 1580 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480: loading cluster ID for follower catalog manager: success
I20260929 10:59:37.255262 1580 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480: acquiring CA information for follower catalog manager: success
I20260929 10:59:37.255577 1580 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260929 10:59:37.297127 1491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:37.297433 1491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:37.297466 1491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:37.299085 1491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:37.299134 1491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:37.299171 1491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.193
I20260929 10:59:37.301266 1491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:41445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.193.keytab
--principal=oryx/127.26.120.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.193:42371
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.120.193
--webserver_port=41851
--tserver_master_addrs=127.26.120.254:34785,127.26.120.253:33873,127.26.120.252:36809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.193
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:37.301501 1491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:37.301734 1491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:37.306986 1588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:37.306995 1586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:37.307217 1491 server_base.cc:1034] running on GCE node
W20260929 10:59:37.307413 1585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:37.307643 1491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:37.307935 1491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:37.309083 1491 hybrid_clock.cc:648] HybridClock initialized: now 1790679577309074 us; error 33 us; skew 500 ppm
Sep 29 10:59:37 dist-test-slave-2lzq krb5kdc[702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679577, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:37.311628 1491 init.cc:377] Logged in from keytab as oryx/127.26.120.193@KRBTEST.COM (short username oryx)
I20260929 10:59:37.312084 1491 webserver.cc:460] Webserver started at http://127.26.120.193:41851/ using document root <none> and password file <none>
I20260929 10:59:37.312361 1491 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:37.312422 1491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:37.313807 1491 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:59:37.314512 1595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:37.314716 1491 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 10:59:37.314813 1491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-0/wal
uuid: "deddd697abc04e13b689f98668eb1250"
format_stamp: "Formatted at 2026-09-29 10:59:36 on dist-test-slave-2lzq"
server_key: "be50769201a49c5adc7efc41112ff0dd"
server_key_iv: "78d7fb123a3f7b8b98839496102c1a87"
server_key_version: "encryptionkey@0"
I20260929 10:59:37.315172 1491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:37.375901 1491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:37.378377 1491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:37.378556 1491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:37.378811 1491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:59:37.379184 1491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:37.379243 1491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:37.379287 1491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:37.379309 1491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:37.386337 1491 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.193:42371
I20260929 10:59:37.386392 1708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.193:42371 every 8 connection(s)
I20260929 10:59:37.386790 1491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
I20260929 10:59:37.393478 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 1491
I20260929 10:59:37.393653 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 943
Sep 29 10:59:37 dist-test-slave-2lzq krb5kdc[702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679577, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.253@KRBTEST.COM
Sep 29 10:59:37 dist-test-slave-2lzq krb5kdc[702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679577, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM
Sep 29 10:59:37 dist-test-slave-2lzq krb5kdc[702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679577, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.252@KRBTEST.COM
I20260929 10:59:37.398924 1395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.387573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:33051 (local address 127.26.120.253:33873)
0929 10:59:37.387864 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:37.387866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:37.387986 (+ 120us) server_negotiation.cc:407] Connection header received
0929 10:59:37.389068 (+ 1082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:37.389069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:37.389084 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:37.389115 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:37.390033 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.390793 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.392228 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.392424 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.392491 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.395473 (+ 2982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:37.395475 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:37.395476 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:37.395484 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:37.397200 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.397713 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.397714 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.397714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.397768 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.398142 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.398143 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.398143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.398307 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:37.398356 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:37.398751 (+ 395us) server_negotiation.cc:299] Negotiation successful
0929 10:59:37.398830 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263}
I20260929 10:59:37.399137 1713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.387597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.253:33873 (local address 127.26.120.193:33051)
0929 10:59:37.387834 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:37.387881 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:37.388950 (+ 1069us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:37.389354 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:37.389354 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:37.389726 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:37.389930 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.389932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.390903 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.390904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.392130 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.392131 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.392566 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.392566 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.392699 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.393420 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:37.393426 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:37.395358 (+ 1932us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:37.397292 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:37.397313 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:37.397316 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:37.397632 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:37.397844 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:37.397845 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:37.397845 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:37.397999 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:37.398435 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:37.398441 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:37.398598 (+ 157us) client_negotiation.cc:769] Sending connection context
0929 10:59:37.398719 (+ 121us) client_negotiation.cc:240] Negotiation successful
0929 10:59:37.398888 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":81,"thread_start_us":85,"threads_started":1}
I20260929 10:59:37.399883 1711 heartbeater.cc:347] Connected to a master server at 127.26.120.253:33873
I20260929 10:59:37.400008 1711 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:37.400223 1711 heartbeater.cc:511] Master 127.26.120.253:33873 requested a full tablet report, sending...
I20260929 10:59:37.400794 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.194:44103
--local_ip_for_outbound_sockets=127.26.120.194
--tserver_master_addrs=127.26.120.254:34785
--webserver_port=42179
--webserver_interface=127.26.120.194
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.194.keytab
--principal=oryx/127.26.120.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:41445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.26.120.254:34785,127.26.120.253:33873
--tserver_master_addrs=127.26.120.254:34785,127.26.120.253:33873,127.26.120.252:36809 with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:59:37.400941 786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.387572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:51969 (local address 127.26.120.254:34785)
0929 10:59:37.387621 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:37.387624 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:37.387963 (+ 339us) server_negotiation.cc:407] Connection header received
0929 10:59:37.389074 (+ 1111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:37.389076 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:37.389088 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:37.389135 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:37.389979 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.390793 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.392209 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.392499 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.392561 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.397650 (+ 5089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:37.397651 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:37.397651 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:37.397660 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:37.399770 (+ 2110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.400150 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.400151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.400152 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.400208 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.400397 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.400397 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.400398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.400544 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:37.400601 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:37.400776 (+ 175us) server_negotiation.cc:299] Negotiation successful
0929 10:59:37.400850 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 10:59:37.401111 1714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.387597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34785 (local address 127.26.120.193:51969)
0929 10:59:37.387858 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:37.387886 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:37.388924 (+ 1038us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:37.389283 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:37.389286 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:37.389673 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:37.389876 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.389880 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.390903 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.390904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.392122 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.392125 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.392562 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.392563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.392696 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.393306 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:37.393317 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:37.397545 (+ 4228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:37.399852 (+ 2307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:37.399853 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:37.399853 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:37.400080 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:37.400278 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:37.400279 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:37.400279 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:37.400338 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:37.400601 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:37.400602 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:37.400644 (+ 42us) client_negotiation.cc:769] Sending connection context
0929 10:59:37.400746 (+ 102us) client_negotiation.cc:240] Negotiation successful
0929 10:59:37.400850 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":10,"spinlock_wait_cycles":384,"thread_start_us":176,"threads_started":1}
I20260929 10:59:37.401437 1710 heartbeater.cc:347] Connected to a master server at 127.26.120.254:34785
I20260929 10:59:37.401475 1710 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:37.401566 1710 heartbeater.cc:511] Master 127.26.120.254:34785 requested a full tablet report, sending...
I20260929 10:59:37.401716 1715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.387956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:36809 (local address 127.26.120.193:60937)
0929 10:59:37.388105 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:37.388114 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:37.388950 (+ 836us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:37.389283 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:37.389287 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:37.389699 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:37.389885 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.389887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.390900 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.390902 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.391966 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.391968 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.392514 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.392515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.392699 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.394986 (+ 2287us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:37.394991 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:37.397211 (+ 2220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:37.400505 (+ 3294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:37.400506 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:37.400506 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:37.400749 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:37.400982 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:37.400982 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:37.400983 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:37.401042 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:37.401321 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:37.401322 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:37.401363 (+ 41us) client_negotiation.cc:769] Sending connection context
0929 10:59:37.401411 (+ 48us) client_negotiation.cc:240] Negotiation successful
0929 10:59:37.401538 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":83,"thread_start_us":102,"threads_started":1}
I20260929 10:59:37.401865 747 ts_manager.cc:194] Re-registered known tserver with Master: deddd697abc04e13b689f98668eb1250 (127.26.120.193:42371)
I20260929 10:59:37.402155 1716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.388169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:60937 (local address 127.26.120.252:36809)
0929 10:59:37.388276 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:37.388278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:37.388286 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:59:37.389071 (+ 785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:37.389073 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:37.389087 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:37.389192 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:37.390012 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.390714 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.392086 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.392404 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.392475 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.397342 (+ 4867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:37.397344 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:37.397345 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:37.397353 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:37.400391 (+ 3038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.400839 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.400840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.400840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.400901 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.401108 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.401109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.401109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.401251 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:37.401828 (+ 577us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:37.401875 (+ 47us) server_negotiation.cc:299] Negotiation successful
0929 10:59:37.401974 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":60,"threads_started":1}
I20260929 10:59:37.402338 747 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.120.193@KRBTEST.COM'} at 127.26.120.193:51969
I20260929 10:59:37.402415 1344 ts_manager.cc:194] Registered new tserver with Master: deddd697abc04e13b689f98668eb1250 (127.26.120.193:42371)
I20260929 10:59:37.402477 1709 heartbeater.cc:347] Connected to a master server at 127.26.120.252:36809
I20260929 10:59:37.402521 1709 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:37.402582 1709 heartbeater.cc:511] Master 127.26.120.252:36809 requested a full tablet report, sending...
I20260929 10:59:37.402900 1530 ts_manager.cc:194] Registered new tserver with Master: deddd697abc04e13b689f98668eb1250 (127.26.120.193:42371)
W20260929 10:59:37.484056 1717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:37.484267 1717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:37.484290 1717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:37.485499 1717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:37.485543 1717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:37.485600 1717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.194
I20260929 10:59:37.487129 1717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:41445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.194.keytab
--principal=oryx/127.26.120.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.194:44103
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.120.194
--webserver_port=42179
--tserver_master_addrs=127.26.120.254:34785,127.26.120.253:33873,127.26.120.252:36809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.194
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:37.487385 1717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:37.487648 1717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:37.492782 1725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:37.492774 1723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:37.492944 1717 server_base.cc:1034] running on GCE node
W20260929 10:59:37.492795 1722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:37.493258 1717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:37.493461 1717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:37.494594 1717 hybrid_clock.cc:648] HybridClock initialized: now 1790679577494567 us; error 40 us; skew 500 ppm
Sep 29 10:59:37 dist-test-slave-2lzq krb5kdc[702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679577, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:37.500384 1717 init.cc:377] Logged in from keytab as oryx/127.26.120.194@KRBTEST.COM (short username oryx)
I20260929 10:59:37.500864 1717 webserver.cc:460] Webserver started at http://127.26.120.194:42179/ using document root <none> and password file <none>
I20260929 10:59:37.501101 1717 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:37.501184 1717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:37.502604 1717 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 10:59:37.503207 1732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:37.503396 1717 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:59:37.503491 1717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-1/wal
uuid: "b586e21fea2f46c385b5f49e297018b7"
format_stamp: "Formatted at 2026-09-29 10:59:36 on dist-test-slave-2lzq"
server_key: "1b240428a87574cd6bca1ac2e6d47121"
server_key_iv: "5ac0bcc67f374bed9794990cf5924492"
server_key_version: "encryptionkey@0"
I20260929 10:59:37.503824 1717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:37.514477 1717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:37.516675 1717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:37.516782 1717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:37.517035 1717 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:59:37.517349 1717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:37.517397 1717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:37.517421 1717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:37.517433 1717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:37.523511 1717 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.194:44103
I20260929 10:59:37.523567 1845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.194:44103 every 8 connection(s)
I20260929 10:59:37.523911 1717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
I20260929 10:59:37.526487 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 1717
I20260929 10:59:37.526660 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 1077
Sep 29 10:59:37 dist-test-slave-2lzq krb5kdc[702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679577, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.253@KRBTEST.COM
Sep 29 10:59:37 dist-test-slave-2lzq krb5kdc[702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679577, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM
I20260929 10:59:37.533633 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.195:43039
--local_ip_for_outbound_sockets=127.26.120.195
--tserver_master_addrs=127.26.120.254:34785
--webserver_port=44259
--webserver_interface=127.26.120.195
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.195.keytab
--principal=oryx/127.26.120.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:41445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.26.120.254:34785,127.26.120.253:33873
--tserver_master_addrs=127.26.120.254:34785,127.26.120.253:33873,127.26.120.252:36809 with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 10:59:37 dist-test-slave-2lzq krb5kdc[702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679577, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.252@KRBTEST.COM
I20260929 10:59:37.535772 1395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.524571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:36793 (local address 127.26.120.253:33873)
0929 10:59:37.524634 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:37.524637 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:37.524886 (+ 249us) server_negotiation.cc:407] Connection header received
0929 10:59:37.525515 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:37.525516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:37.525529 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:37.525562 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:37.526353 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.527138 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.528729 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.528936 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.529014 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.532384 (+ 3370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:37.532385 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:37.532386 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:37.532393 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:37.534262 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.534935 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.534936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.534937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.534996 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.535213 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.535215 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.535215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.535347 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:37.535409 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:37.535602 (+ 193us) server_negotiation.cc:299] Negotiation successful
0929 10:59:37.535685 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260929 10:59:37.536007 1852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.524870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34785 (local address 127.26.120.194:51583)
0929 10:59:37.525059 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:37.525069 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:37.525421 (+ 352us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:37.525662 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:37.525663 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:37.525976 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:37.526212 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.526218 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.527271 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.527272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.528617 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.528620 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.529138 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.529139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.529212 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.529947 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:37.529951 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:37.532275 (+ 2324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:37.534205 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:37.534211 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:37.534214 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:37.534532 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:37.534757 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:37.534758 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:37.534758 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:37.534892 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:37.535231 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:37.535238 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:37.535403 (+ 165us) client_negotiation.cc:769] Sending connection context
0929 10:59:37.535548 (+ 145us) client_negotiation.cc:240] Negotiation successful
0929 10:59:37.535717 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":60,"thread_start_us":56,"threads_started":1}
I20260929 10:59:37.536074 1850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.524572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.253:33873 (local address 127.26.120.194:36793)
0929 10:59:37.524765 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:37.524808 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:37.525418 (+ 610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:37.526073 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:37.526075 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:37.526214 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:37.526267 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.526269 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.527271 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.527272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.528617 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.528620 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.529033 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.529034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.529213 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.529891 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:37.529901 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:37.532275 (+ 2374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:37.534580 (+ 2305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:37.534581 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:37.534582 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:37.534842 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:37.535071 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:37.535072 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:37.535072 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:37.535126 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:37.535437 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:37.535437 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:37.535473 (+ 36us) client_negotiation.cc:769] Sending connection context
0929 10:59:37.535858 (+ 385us) client_negotiation.cc:240] Negotiation successful
0929 10:59:37.535950 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":48,"thread_start_us":93,"threads_started":1}
I20260929 10:59:37.536453 1848 heartbeater.cc:347] Connected to a master server at 127.26.120.253:33873
I20260929 10:59:37.536571 1848 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:37.536597 786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.524953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:51583 (local address 127.26.120.254:34785)
0929 10:59:37.524996 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:37.524998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:37.525124 (+ 126us) server_negotiation.cc:407] Connection header received
0929 10:59:37.525515 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:37.525516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:37.525529 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:37.525570 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:37.526325 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.527147 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.528703 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.529061 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.529132 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.532384 (+ 3252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:37.532385 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:37.532386 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:37.532394 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:37.534113 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.534623 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.534625 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.534625 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.534687 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.534966 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.534967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.534968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.535137 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:37.535182 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:37.535582 (+ 400us) server_negotiation.cc:299] Negotiation successful
0929 10:59:37.535670 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 10:59:37.536772 1848 heartbeater.cc:511] Master 127.26.120.253:33873 requested a full tablet report, sending...
I20260929 10:59:37.537087 1847 heartbeater.cc:347] Connected to a master server at 127.26.120.254:34785
I20260929 10:59:37.537171 1847 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:37.537241 1344 ts_manager.cc:194] Registered new tserver with Master: b586e21fea2f46c385b5f49e297018b7 (127.26.120.194:44103)
I20260929 10:59:37.537286 1847 heartbeater.cc:511] Master 127.26.120.254:34785 requested a full tablet report, sending...
I20260929 10:59:37.537612 747 ts_manager.cc:194] Re-registered known tserver with Master: b586e21fea2f46c385b5f49e297018b7 (127.26.120.194:44103)
I20260929 10:59:37.537961 1716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.524677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:37363 (local address 127.26.120.252:36809)
0929 10:59:37.524719 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:37.524721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:37.524908 (+ 187us) server_negotiation.cc:407] Connection header received
0929 10:59:37.525507 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:37.525509 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:37.525525 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:37.525576 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:37.526307 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.527121 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.531495 (+ 4374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.531843 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.531918 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.535011 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:37.535012 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:37.535013 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:37.535021 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:37.536714 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.537118 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.537120 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.537120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.537178 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.537400 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.537401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.537402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.537533 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:37.537598 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:37.537784 (+ 186us) server_negotiation.cc:299] Negotiation successful
0929 10:59:37.537872 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 10:59:37.538121 747 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.120.194@KRBTEST.COM'} at 127.26.120.194:51583
I20260929 10:59:37.538492 1851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.524572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:36809 (local address 127.26.120.194:37363)
0929 10:59:37.524770 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:37.524807 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:37.525397 (+ 590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:37.525641 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:37.525645 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:37.525954 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:37.526211 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.526215 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.530403 (+ 4188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.530405 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.531391 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.531394 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.531930 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.531931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.532016 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.532455 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:37.532460 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:37.534909 (+ 2449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:37.536804 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:37.536805 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:37.536805 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:37.537032 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:37.537249 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:37.537250 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:37.537250 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:37.537324 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:37.537631 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:37.537632 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:37.537675 (+ 43us) client_negotiation.cc:769] Sending connection context
0929 10:59:37.538041 (+ 366us) client_negotiation.cc:240] Negotiation successful
0929 10:59:37.538165 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":6,"spinlock_wait_cycles":7936,"thread_start_us":129,"threads_started":1}
I20260929 10:59:37.538874 1846 heartbeater.cc:347] Connected to a master server at 127.26.120.252:36809
I20260929 10:59:37.538933 1846 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:37.539012 1846 heartbeater.cc:511] Master 127.26.120.252:36809 requested a full tablet report, sending...
I20260929 10:59:37.539321 1530 ts_manager.cc:194] Registered new tserver with Master: b586e21fea2f46c385b5f49e297018b7 (127.26.120.194:44103)
W20260929 10:59:37.617065 1853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:37.617352 1853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:37.617379 1853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:37.618736 1853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:37.618780 1853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:37.618808 1853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.195
I20260929 10:59:37.620225 1853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:41445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.195.keytab
--principal=oryx/127.26.120.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.195:43039
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.120.195
--webserver_port=44259
--tserver_master_addrs=127.26.120.254:34785,127.26.120.253:33873,127.26.120.252:36809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.195
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:37.620409 1853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:37.620658 1853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:37.625748 1858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:37.625758 1859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:37.625775 1853 server_base.cc:1034] running on GCE node
W20260929 10:59:37.625764 1861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:37.626170 1853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:37.626448 1853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:37.627619 1853 hybrid_clock.cc:648] HybridClock initialized: now 1790679577627608 us; error 25 us; skew 500 ppm
Sep 29 10:59:37 dist-test-slave-2lzq krb5kdc[702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679577, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:37.630187 1853 init.cc:377] Logged in from keytab as oryx/127.26.120.195@KRBTEST.COM (short username oryx)
I20260929 10:59:37.630607 1853 webserver.cc:460] Webserver started at http://127.26.120.195:44259/ using document root <none> and password file <none>
I20260929 10:59:37.630842 1853 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:37.630923 1853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:37.632222 1853 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:59:37.632802 1868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:37.632977 1853 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:59:37.633077 1853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-2/wal
uuid: "83065b57befe412fbf130152240e8700"
format_stamp: "Formatted at 2026-09-29 10:59:36 on dist-test-slave-2lzq"
server_key: "31bdc1a37741a1188c307956500136d2"
server_key_iv: "63b63cbb47511c82a7a3dd0be6bc9f67"
server_key_version: "encryptionkey@0"
I20260929 10:59:37.633435 1853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:37.652243 1853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:37.654287 1853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:37.654448 1853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:37.654670 1853 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:59:37.654963 1853 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:37.655014 1853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:37.655050 1853 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:37.655076 1853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:37.660926 1853 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.195:43039
I20260929 10:59:37.660991 1981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.195:43039 every 8 connection(s)
I20260929 10:59:37.661300 1853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
Sep 29 10:59:37 dist-test-slave-2lzq krb5kdc[702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679577, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.253@KRBTEST.COM
Sep 29 10:59:37 dist-test-slave-2lzq krb5kdc[702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679577, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.252@KRBTEST.COM
Sep 29 10:59:37 dist-test-slave-2lzq krb5kdc[702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679577, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM
I20260929 10:59:37.671443 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 1853
I20260929 10:59:37.672140 1716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.662000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:55255 (local address 127.26.120.252:36809)
0929 10:59:37.662118 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:37.662121 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:37.662320 (+ 199us) server_negotiation.cc:407] Connection header received
0929 10:59:37.663394 (+ 1074us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:37.663395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:37.663408 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:37.663441 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:37.664185 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.664934 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.666588 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.666889 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.666953 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.669316 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:37.669317 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:37.669317 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:37.669323 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:37.670895 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.671272 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.671273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.671273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.671328 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.671521 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.671522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.671522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.671654 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:37.671694 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:37.671948 (+ 254us) server_negotiation.cc:299] Negotiation successful
0929 10:59:37.672055 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260929 10:59:37.672490 1986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.662004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:36809 (local address 127.26.120.195:55255)
0929 10:59:37.662205 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:37.662245 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:37.663272 (+ 1027us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:37.663518 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:37.663521 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:37.663883 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:37.664076 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.664083 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.665010 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.665011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.666079 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.666081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.666962 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.666963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.667078 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.667441 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:37.667445 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:37.669228 (+ 1783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:37.670963 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:37.670964 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:37.670965 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:37.671166 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:37.671399 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:37.671400 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:37.671400 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:37.671450 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:37.671735 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:37.671736 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:37.671802 (+ 66us) client_negotiation.cc:769] Sending connection context
0929 10:59:37.672223 (+ 421us) client_negotiation.cc:240] Negotiation successful
0929 10:59:37.672326 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":46,"thread_start_us":138,"threads_started":1}
I20260929 10:59:37.672674 1395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.661983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:56259 (local address 127.26.120.253:33873)
0929 10:59:37.662041 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:37.662043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:37.662345 (+ 302us) server_negotiation.cc:407] Connection header received
0929 10:59:37.663365 (+ 1020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:37.663367 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:37.663379 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:37.663438 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:37.664180 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.664669 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.666162 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.666456 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.666530 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.668903 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:37.668904 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:37.668905 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:37.668912 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:37.670547 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.670998 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.670999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.670999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.671064 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.671357 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.671358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.671358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.671494 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:37.671567 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:37.672470 (+ 903us) server_negotiation.cc:299] Negotiation successful
0929 10:59:37.672561 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260929 10:59:37.673331 786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.662446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:56925 (local address 127.26.120.254:34785)
0929 10:59:37.662497 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:37.662499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:37.662515 (+ 16us) server_negotiation.cc:407] Connection header received
0929 10:59:37.663393 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:37.663394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:37.663408 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:37.663441 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:37.664184 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.664933 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.667081 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.667361 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.667433 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.670495 (+ 3062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:37.670496 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:37.670496 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:37.670503 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:37.672224 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.672555 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.672557 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.672557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.672616 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.672817 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.672818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.672818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.672929 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:37.673005 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:37.673157 (+ 152us) server_negotiation.cc:299] Negotiation successful
0929 10:59:37.673233 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260929 10:59:37.673362 1987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.662004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.253:33873 (local address 127.26.120.195:56259)
0929 10:59:37.662206 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:37.662245 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:37.663269 (+ 1024us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:37.663517 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:37.663521 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:37.663880 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:37.664075 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.664079 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.664768 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.664769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.666079 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.666081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.666545 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.666546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.666694 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.667187 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:37.667196 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:37.668822 (+ 1626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:37.670637 (+ 1815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:37.670643 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:37.670645 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:37.670924 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:37.671132 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:37.671132 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:37.671133 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:37.671280 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:37.671644 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:37.671651 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:37.671802 (+ 151us) client_negotiation.cc:769] Sending connection context
0929 10:59:37.671905 (+ 103us) client_negotiation.cc:240] Negotiation successful
0929 10:59:37.672074 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":40,"thread_start_us":126,"threads_started":1}
I20260929 10:59:37.673395 1988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.662332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34785 (local address 127.26.120.195:56925)
0929 10:59:37.662460 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:37.662469 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:37.663281 (+ 812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:37.663518 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:37.663521 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:37.663890 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:37.664076 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.664083 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.666150 (+ 2067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.666151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.666991 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.666993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.667432 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.667433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.667494 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.668998 (+ 1504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:37.669003 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:37.670410 (+ 1407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:37.672315 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:37.672316 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:37.672317 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:37.672476 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:37.672702 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:37.672703 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:37.672703 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:37.672766 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:37.672995 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:37.672995 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:37.673041 (+ 46us) client_negotiation.cc:769] Sending connection context
0929 10:59:37.673129 (+ 88us) client_negotiation.cc:240] Negotiation successful
0929 10:59:37.673235 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":5,"thread_start_us":77,"threads_started":1}
I20260929 10:59:37.673560 1982 heartbeater.cc:347] Connected to a master server at 127.26.120.252:36809
I20260929 10:59:37.673638 1982 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:37.673816 1982 heartbeater.cc:511] Master 127.26.120.252:36809 requested a full tablet report, sending...
I20260929 10:59:37.673895 1983 heartbeater.cc:347] Connected to a master server at 127.26.120.254:34785
I20260929 10:59:37.673933 1983 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:37.673987 1983 heartbeater.cc:511] Master 127.26.120.254:34785 requested a full tablet report, sending...
I20260929 10:59:37.674224 747 ts_manager.cc:194] Re-registered known tserver with Master: 83065b57befe412fbf130152240e8700 (127.26.120.195:43039)
I20260929 10:59:37.674347 1984 heartbeater.cc:347] Connected to a master server at 127.26.120.253:33873
I20260929 10:59:37.674408 1984 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:37.674522 1984 heartbeater.cc:511] Master 127.26.120.253:33873 requested a full tablet report, sending...
I20260929 10:59:37.674763 747 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.120.195@KRBTEST.COM'} at 127.26.120.195:56925
I20260929 10:59:37.674849 1344 ts_manager.cc:194] Registered new tserver with Master: 83065b57befe412fbf130152240e8700 (127.26.120.195:43039)
I20260929 10:59:37.675007 1530 ts_manager.cc:194] Registered new tserver with Master: 83065b57befe412fbf130152240e8700 (127.26.120.195:43039)
I20260929 10:59:37.682626 1716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.674694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48276 (local address 127.26.120.252:36809)
0929 10:59:37.674726 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:37.674728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:37.675428 (+ 700us) server_negotiation.cc:407] Connection header received
0929 10:59:37.675489 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:37.675490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:37.675501 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:37.675555 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:37.676272 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.676839 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.677780 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.678098 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.678155 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.679595 (+ 1440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:37.679597 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:37.679597 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:37.679604 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:37.681278 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.681728 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.681729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.681730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.681781 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.682005 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.682007 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.682007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.682161 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:37.682256 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:37.682447 (+ 191us) server_negotiation.cc:299] Negotiation successful
0929 10:59:37.682546 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260929 10:59:37.682624 1395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.674259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43490 (local address 127.26.120.253:33873)
0929 10:59:37.674882 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:37.674884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:37.675580 (+ 696us) server_negotiation.cc:407] Connection header received
0929 10:59:37.675671 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:37.675671 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:37.675681 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:37.675710 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:37.676381 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.676963 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.677696 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.677975 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.678038 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.679786 (+ 1748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:37.679787 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:37.679787 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:37.679793 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:37.681252 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.681685 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.681686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.681687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.681745 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.681921 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.681922 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.681923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.682093 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:37.682178 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:37.682458 (+ 280us) server_negotiation.cc:299] Negotiation successful
0929 10:59:37.682531 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600}
I20260929 10:59:37.682888 786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.674485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60542 (local address 127.26.120.254:34785)
0929 10:59:37.674547 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:37.674548 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:37.675267 (+ 719us) server_negotiation.cc:407] Connection header received
0929 10:59:37.675329 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:37.675330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:37.675341 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:37.675392 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:37.676061 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.676651 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.677723 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.677962 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.677993 (+ 31us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.679996 (+ 2003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:37.679997 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:37.679998 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:37.680004 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:37.681600 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.681952 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.681954 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.681954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.682023 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.682261 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.682262 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.682262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.682370 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:37.682437 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:37.682665 (+ 228us) server_negotiation.cc:299] Negotiation successful
0929 10:59:37.682781 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 10:59:37.684481 747 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60542:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:59:37.686195 747 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:59:37.693759 2001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.689875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:42371 (local address 127.0.0.1:44578)
0929 10:59:37.689975 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:37.689988 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:37.690361 (+ 373us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:37.690636 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:37.690637 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:37.690640 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:37.690690 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.690692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.691796 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.691797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.692763 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.692765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.693496 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.693497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.693564 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.693567 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:59:37.693626 (+ 59us) client_negotiation.cc:240] Negotiation successful
0929 10:59:37.693682 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69,"thread_start_us":53,"threads_started":1}
I20260929 10:59:37.693769 2004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.690258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44578 (local address 127.26.120.193:42371)
0929 10:59:37.690430 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:37.690437 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:37.690448 (+ 11us) server_negotiation.cc:407] Connection header received
0929 10:59:37.690493 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:37.690495 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:37.690512 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:37.690769 (+ 257us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:37.690834 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.691700 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.692857 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.693383 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.693517 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.693587 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:37.693644 (+ 57us) server_negotiation.cc:299] Negotiation successful
0929 10:59:37.693685 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":93,"threads_started":1}
I20260929 10:59:37.694756 2003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.690016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:43039 (local address 127.0.0.1:37540)
0929 10:59:37.690192 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:37.690202 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:37.690260 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:37.690969 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:37.690970 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:37.690972 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:37.691022 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.691024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.692492 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.692493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.693561 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.693564 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.694567 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.694568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.694640 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.694644 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:59:37.694679 (+ 35us) client_negotiation.cc:240] Negotiation successful
0929 10:59:37.694708 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":104,"threads_started":1}
I20260929 10:59:37.694844 1643 tablet_service.cc:1467] Processing CreateTablet for tablet 780ac9a706e040fbabacc78a45de0ec1 (DEFAULT_TABLE table=test-table [id=a36ca24a4c65464c904bb6cc53c963c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:37.695116 2006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.690569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37540 (local address 127.26.120.195:43039)
0929 10:59:37.690804 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:37.690809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:37.690819 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:59:37.690852 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:37.690854 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:37.690868 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:37.690947 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:37.691105 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.692394 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.693652 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.694413 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.694895 (+ 482us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.694954 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:37.694982 (+ 28us) server_negotiation.cc:299] Negotiation successful
0929 10:59:37.695033 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":91,"threads_started":1}
I20260929 10:59:37.695240 2005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.690385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48720 (local address 127.26.120.194:44103)
0929 10:59:37.690551 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:37.690555 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:37.690562 (+ 7us) server_negotiation.cc:407] Connection header received
0929 10:59:37.690592 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:37.690594 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:37.690605 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:37.690690 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:37.690845 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.691957 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.693360 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.694098 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.694162 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.694210 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:37.695116 (+ 906us) server_negotiation.cc:299] Negotiation successful
0929 10:59:37.695163 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":92,"threads_started":1}
I20260929 10:59:37.695304 1643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 780ac9a706e040fbabacc78a45de0ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:37.695377 2002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.689876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:44103 (local address 127.0.0.1:48720)
0929 10:59:37.690015 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:37.690024 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:37.690104 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:37.690700 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:37.690700 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:37.690703 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:37.690758 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.690760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.692073 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.692074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.693249 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.693252 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.694711 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.694712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.694784 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.694788 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:59:37.695099 (+ 311us) client_negotiation.cc:240] Negotiation successful
0929 10:59:37.695147 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":90,"threads_started":1}
I20260929 10:59:37.698140 2007 tablet_bootstrap.cc:492] T 780ac9a706e040fbabacc78a45de0ec1 P deddd697abc04e13b689f98668eb1250: Bootstrap starting.
I20260929 10:59:37.698773 2007 tablet_bootstrap.cc:654] T 780ac9a706e040fbabacc78a45de0ec1 P deddd697abc04e13b689f98668eb1250: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:37.698760 1916 tablet_service.cc:1467] Processing CreateTablet for tablet 780ac9a706e040fbabacc78a45de0ec1 (DEFAULT_TABLE table=test-table [id=a36ca24a4c65464c904bb6cc53c963c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:37.698879 1780 tablet_service.cc:1467] Processing CreateTablet for tablet 780ac9a706e040fbabacc78a45de0ec1 (DEFAULT_TABLE table=test-table [id=a36ca24a4c65464c904bb6cc53c963c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:37.699117 2007 log.cc:824] T 780ac9a706e040fbabacc78a45de0ec1 P deddd697abc04e13b689f98668eb1250: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:37.699174 1780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 780ac9a706e040fbabacc78a45de0ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:37.699167 1916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 780ac9a706e040fbabacc78a45de0ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:37.699836 2007 tablet_bootstrap.cc:492] T 780ac9a706e040fbabacc78a45de0ec1 P deddd697abc04e13b689f98668eb1250: No bootstrap required, opened a new log
I20260929 10:59:37.699921 2007 ts_tablet_manager.cc:1397] T 780ac9a706e040fbabacc78a45de0ec1 P deddd697abc04e13b689f98668eb1250: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:59:37.701911 2007 raft_consensus.cc:348] T 780ac9a706e040fbabacc78a45de0ec1 P deddd697abc04e13b689f98668eb1250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deddd697abc04e13b689f98668eb1250" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42371 } } peers { permanent_uuid: "83065b57befe412fbf130152240e8700" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43039 } } peers { permanent_uuid: "b586e21fea2f46c385b5f49e297018b7" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 44103 } }
I20260929 10:59:37.702067 2007 raft_consensus.cc:374] T 780ac9a706e040fbabacc78a45de0ec1 P deddd697abc04e13b689f98668eb1250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:37.702144 2007 raft_consensus.cc:729] T 780ac9a706e040fbabacc78a45de0ec1 P deddd697abc04e13b689f98668eb1250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: deddd697abc04e13b689f98668eb1250, State: Initialized, Role: FOLLOWER
I20260929 10:59:37.702258 2007 consensus_queue.cc:260] T 780ac9a706e040fbabacc78a45de0ec1 P deddd697abc04e13b689f98668eb1250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deddd697abc04e13b689f98668eb1250" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42371 } } peers { permanent_uuid: "83065b57befe412fbf130152240e8700" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43039 } } peers { permanent_uuid: "b586e21fea2f46c385b5f49e297018b7" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 44103 } }
I20260929 10:59:37.702525 2007 ts_tablet_manager.cc:1428] T 780ac9a706e040fbabacc78a45de0ec1 P deddd697abc04e13b689f98668eb1250: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260929 10:59:37.702677 1710 heartbeater.cc:503] Master 127.26.120.254:34785 was elected leader, sending a full tablet report...
I20260929 10:59:37.703828 2011 tablet_bootstrap.cc:492] T 780ac9a706e040fbabacc78a45de0ec1 P 83065b57befe412fbf130152240e8700: Bootstrap starting.
I20260929 10:59:37.704187 2010 tablet_bootstrap.cc:492] T 780ac9a706e040fbabacc78a45de0ec1 P b586e21fea2f46c385b5f49e297018b7: Bootstrap starting.
I20260929 10:59:37.704738 2010 tablet_bootstrap.cc:654] T 780ac9a706e040fbabacc78a45de0ec1 P b586e21fea2f46c385b5f49e297018b7: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:37.704972 2010 log.cc:824] T 780ac9a706e040fbabacc78a45de0ec1 P b586e21fea2f46c385b5f49e297018b7: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:37.705523 2011 tablet_bootstrap.cc:654] T 780ac9a706e040fbabacc78a45de0ec1 P 83065b57befe412fbf130152240e8700: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:37.705760 2010 tablet_bootstrap.cc:492] T 780ac9a706e040fbabacc78a45de0ec1 P b586e21fea2f46c385b5f49e297018b7: No bootstrap required, opened a new log
I20260929 10:59:37.705829 2010 ts_tablet_manager.cc:1397] T 780ac9a706e040fbabacc78a45de0ec1 P b586e21fea2f46c385b5f49e297018b7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:59:37.705866 2011 log.cc:824] T 780ac9a706e040fbabacc78a45de0ec1 P 83065b57befe412fbf130152240e8700: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:37.706668 2011 tablet_bootstrap.cc:492] T 780ac9a706e040fbabacc78a45de0ec1 P 83065b57befe412fbf130152240e8700: No bootstrap required, opened a new log
I20260929 10:59:37.706733 2011 ts_tablet_manager.cc:1397] T 780ac9a706e040fbabacc78a45de0ec1 P 83065b57befe412fbf130152240e8700: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260929 10:59:37.708647 2010 raft_consensus.cc:348] T 780ac9a706e040fbabacc78a45de0ec1 P b586e21fea2f46c385b5f49e297018b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deddd697abc04e13b689f98668eb1250" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42371 } } peers { permanent_uuid: "83065b57befe412fbf130152240e8700" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43039 } } peers { permanent_uuid: "b586e21fea2f46c385b5f49e297018b7" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 44103 } }
I20260929 10:59:37.708837 2010 raft_consensus.cc:374] T 780ac9a706e040fbabacc78a45de0ec1 P b586e21fea2f46c385b5f49e297018b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:37.708884 2010 raft_consensus.cc:729] T 780ac9a706e040fbabacc78a45de0ec1 P b586e21fea2f46c385b5f49e297018b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b586e21fea2f46c385b5f49e297018b7, State: Initialized, Role: FOLLOWER
I20260929 10:59:37.709025 2010 consensus_queue.cc:260] T 780ac9a706e040fbabacc78a45de0ec1 P b586e21fea2f46c385b5f49e297018b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deddd697abc04e13b689f98668eb1250" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42371 } } peers { permanent_uuid: "83065b57befe412fbf130152240e8700" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43039 } } peers { permanent_uuid: "b586e21fea2f46c385b5f49e297018b7" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 44103 } }
I20260929 10:59:37.709417 2010 ts_tablet_manager.cc:1428] T 780ac9a706e040fbabacc78a45de0ec1 P b586e21fea2f46c385b5f49e297018b7: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260929 10:59:37.709450 1847 heartbeater.cc:503] Master 127.26.120.254:34785 was elected leader, sending a full tablet report...
I20260929 10:59:37.710086 2011 raft_consensus.cc:348] T 780ac9a706e040fbabacc78a45de0ec1 P 83065b57befe412fbf130152240e8700 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deddd697abc04e13b689f98668eb1250" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42371 } } peers { permanent_uuid: "83065b57befe412fbf130152240e8700" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43039 } } peers { permanent_uuid: "b586e21fea2f46c385b5f49e297018b7" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 44103 } }
I20260929 10:59:37.710253 2011 raft_consensus.cc:374] T 780ac9a706e040fbabacc78a45de0ec1 P 83065b57befe412fbf130152240e8700 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:37.710296 2011 raft_consensus.cc:729] T 780ac9a706e040fbabacc78a45de0ec1 P 83065b57befe412fbf130152240e8700 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83065b57befe412fbf130152240e8700, State: Initialized, Role: FOLLOWER
I20260929 10:59:37.710445 2011 consensus_queue.cc:260] T 780ac9a706e040fbabacc78a45de0ec1 P 83065b57befe412fbf130152240e8700 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deddd697abc04e13b689f98668eb1250" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42371 } } peers { permanent_uuid: "83065b57befe412fbf130152240e8700" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43039 } } peers { permanent_uuid: "b586e21fea2f46c385b5f49e297018b7" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 44103 } }
I20260929 10:59:37.710741 2011 ts_tablet_manager.cc:1428] T 780ac9a706e040fbabacc78a45de0ec1 P 83065b57befe412fbf130152240e8700: Time spent starting tablet: real 0.004s user 0.002s sys 0.002s
I20260929 10:59:37.710779 1983 heartbeater.cc:503] Master 127.26.120.254:34785 was elected leader, sending a full tablet report...
I20260929 10:59:37.735292 1716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.733037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48288 (local address 127.26.120.252:36809)
0929 10:59:37.733108 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:37.733110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:37.733118 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:59:37.733139 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:37.733140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:37.733156 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:37.733211 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:37.733327 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.733804 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.734688 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.735047 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.735104 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.735154 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:37.735221 (+ 67us) server_negotiation.cc:299] Negotiation successful
0929 10:59:37.735244 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 10:59:37.735291 2002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.732970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:36809 (local address 127.0.0.1:48288)
0929 10:59:37.733029 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:37.733041 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:37.733090 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:37.733222 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:37.733223 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:37.733224 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:37.733274 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.733275 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.733861 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.733861 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.734619 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.734621 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.735114 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.735114 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.735159 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.735163 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:59:37.735202 (+ 39us) client_negotiation.cc:240] Negotiation successful
0929 10:59:37.735234 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260929 10:59:37.735636 1540 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260929 10:59:37.736474 1569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c9a8ea9e35a43a99072b3ab64ca4b8d. Latest consensus state: current_term: 1 leader_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: false } } peers { permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: NON_VOTER last_known_addr { host: "127.26.120.252" port: 36809 } attrs { promote: true } } }
I20260929 10:59:37.736591 1569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [sys.catalog]: This master's current role is: LEARNER
W20260929 10:59:37.774740 1849 tablet.cc:2369] T 780ac9a706e040fbabacc78a45de0ec1 P b586e21fea2f46c385b5f49e297018b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:59:37.852608 2014 raft_consensus.cc:482] T 780ac9a706e040fbabacc78a45de0ec1 P b586e21fea2f46c385b5f49e297018b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:59:37.852794 2014 raft_consensus.cc:504] T 780ac9a706e040fbabacc78a45de0ec1 P b586e21fea2f46c385b5f49e297018b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deddd697abc04e13b689f98668eb1250" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42371 } } peers { permanent_uuid: "83065b57befe412fbf130152240e8700" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43039 } } peers { permanent_uuid: "b586e21fea2f46c385b5f49e297018b7" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 44103 } }
I20260929 10:59:37.853257 2014 leader_election.cc:288] T 780ac9a706e040fbabacc78a45de0ec1 P b586e21fea2f46c385b5f49e297018b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers deddd697abc04e13b689f98668eb1250 (127.26.120.193:42371), 83065b57befe412fbf130152240e8700 (127.26.120.195:43039)
I20260929 10:59:37.856271 1851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.853426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:42371 (local address 127.26.120.194:36917)
0929 10:59:37.853462 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:37.853475 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:37.853520 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:37.853801 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:37.853802 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:37.853804 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:37.853881 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.853883 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.854637 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.854638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.855433 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.855435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.856037 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.856038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.856125 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.856130 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 10:59:37.856183 (+ 53us) client_negotiation.cc:240] Negotiation successful
0929 10:59:37.856221 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15,"mutex_wait_us":30}
I20260929 10:59:37.856344 2004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.853608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:36917 (local address 127.26.120.193:42371)
0929 10:59:37.853657 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:37.853660 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:37.853668 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:59:37.853698 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:37.853699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:37.853709 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:37.853788 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:37.853998 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.854564 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.855530 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.855947 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.856003 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.856010 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:37.856202 (+ 192us) server_negotiation.cc:299] Negotiation successful
0929 10:59:37.856255 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260929 10:59:37.856557 1852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.853557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:43039 (local address 127.26.120.194:34145)
0929 10:59:37.853617 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:37.853629 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:37.853695 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:37.853858 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:37.853858 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:37.853860 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:37.854095 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.854097 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.854720 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.854721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.855583 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:37.855585 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.856177 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.856178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:37.856321 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.856325 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:59:37.856385 (+ 60us) client_negotiation.cc:240] Negotiation successful
0929 10:59:37.856429 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260929 10:59:37.856688 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "780ac9a706e040fbabacc78a45de0ec1" candidate_uuid: "b586e21fea2f46c385b5f49e297018b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "deddd697abc04e13b689f98668eb1250" is_pre_election: true
I20260929 10:59:37.856756 2006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.853633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:34145 (local address 127.26.120.195:43039)
0929 10:59:37.853677 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:37.853679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:37.853688 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:59:37.853774 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:37.853775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:37.853784 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:37.853835 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:37.854165 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.854634 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.855665 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.856068 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.856147 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.856155 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:37.856656 (+ 501us) server_negotiation.cc:299] Negotiation successful
0929 10:59:37.856701 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260929 10:59:37.856834 1663 raft_consensus.cc:2454] T 780ac9a706e040fbabacc78a45de0ec1 P deddd697abc04e13b689f98668eb1250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b586e21fea2f46c385b5f49e297018b7 in term 0.
I20260929 10:59:37.857069 1734 leader_election.cc:302] T 780ac9a706e040fbabacc78a45de0ec1 P b586e21fea2f46c385b5f49e297018b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b586e21fea2f46c385b5f49e297018b7, deddd697abc04e13b689f98668eb1250; no voters:
I20260929 10:59:37.857084 1936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "780ac9a706e040fbabacc78a45de0ec1" candidate_uuid: "b586e21fea2f46c385b5f49e297018b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83065b57befe412fbf130152240e8700" is_pre_election: true
I20260929 10:59:37.857213 1936 raft_consensus.cc:2454] T 780ac9a706e040fbabacc78a45de0ec1 P 83065b57befe412fbf130152240e8700 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b586e21fea2f46c385b5f49e297018b7 in term 0.
I20260929 10:59:37.857213 2014 raft_consensus.cc:2781] T 780ac9a706e040fbabacc78a45de0ec1 P b586e21fea2f46c385b5f49e297018b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:59:37.857276 2014 raft_consensus.cc:482] T 780ac9a706e040fbabacc78a45de0ec1 P b586e21fea2f46c385b5f49e297018b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:59:37.857323 2014 raft_consensus.cc:3037] T 780ac9a706e040fbabacc78a45de0ec1 P b586e21fea2f46c385b5f49e297018b7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:37.858016 2014 raft_consensus.cc:504] T 780ac9a706e040fbabacc78a45de0ec1 P b586e21fea2f46c385b5f49e297018b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deddd697abc04e13b689f98668eb1250" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42371 } } peers { permanent_uuid: "83065b57befe412fbf130152240e8700" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43039 } } peers { permanent_uuid: "b586e21fea2f46c385b5f49e297018b7" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 44103 } }
I20260929 10:59:37.858191 2014 leader_election.cc:288] T 780ac9a706e040fbabacc78a45de0ec1 P b586e21fea2f46c385b5f49e297018b7 [CANDIDATE]: Term 1 election: Requested vote from peers deddd697abc04e13b689f98668eb1250 (127.26.120.193:42371), 83065b57befe412fbf130152240e8700 (127.26.120.195:43039)
I20260929 10:59:37.858312 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "780ac9a706e040fbabacc78a45de0ec1" candidate_uuid: "b586e21fea2f46c385b5f49e297018b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "deddd697abc04e13b689f98668eb1250"
I20260929 10:59:37.858367 1936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "780ac9a706e040fbabacc78a45de0ec1" candidate_uuid: "b586e21fea2f46c385b5f49e297018b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83065b57befe412fbf130152240e8700"
I20260929 10:59:37.858397 1663 raft_consensus.cc:3037] T 780ac9a706e040fbabacc78a45de0ec1 P deddd697abc04e13b689f98668eb1250 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:37.858448 1936 raft_consensus.cc:3037] T 780ac9a706e040fbabacc78a45de0ec1 P 83065b57befe412fbf130152240e8700 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:37.859098 1936 raft_consensus.cc:2454] T 780ac9a706e040fbabacc78a45de0ec1 P 83065b57befe412fbf130152240e8700 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b586e21fea2f46c385b5f49e297018b7 in term 1.
I20260929 10:59:37.859166 1663 raft_consensus.cc:2454] T 780ac9a706e040fbabacc78a45de0ec1 P deddd697abc04e13b689f98668eb1250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b586e21fea2f46c385b5f49e297018b7 in term 1.
I20260929 10:59:37.859329 1734 leader_election.cc:302] T 780ac9a706e040fbabacc78a45de0ec1 P b586e21fea2f46c385b5f49e297018b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b586e21fea2f46c385b5f49e297018b7, deddd697abc04e13b689f98668eb1250; no voters:
I20260929 10:59:37.859433 2014 raft_consensus.cc:2781] T 780ac9a706e040fbabacc78a45de0ec1 P b586e21fea2f46c385b5f49e297018b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:59:37.859604 2014 raft_consensus.cc:686] T 780ac9a706e040fbabacc78a45de0ec1 P b586e21fea2f46c385b5f49e297018b7 [term 1 LEADER]: Becoming Leader. State: Replica: b586e21fea2f46c385b5f49e297018b7, State: Running, Role: LEADER
I20260929 10:59:37.859715 2014 consensus_queue.cc:237] T 780ac9a706e040fbabacc78a45de0ec1 P b586e21fea2f46c385b5f49e297018b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deddd697abc04e13b689f98668eb1250" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42371 } } peers { permanent_uuid: "83065b57befe412fbf130152240e8700" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43039 } } peers { permanent_uuid: "b586e21fea2f46c385b5f49e297018b7" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 44103 } }
I20260929 10:59:37.860432 747 catalog_manager.cc:5495] T 780ac9a706e040fbabacc78a45de0ec1 P b586e21fea2f46c385b5f49e297018b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b586e21fea2f46c385b5f49e297018b7 (127.26.120.194). New cstate: current_term: 1 leader_uuid: "b586e21fea2f46c385b5f49e297018b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deddd697abc04e13b689f98668eb1250" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83065b57befe412fbf130152240e8700" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b586e21fea2f46c385b5f49e297018b7" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 44103 } health_report { overall_health: HEALTHY } } }
I20260929 10:59:37.861121 1496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: false } } peers { permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: NON_VOTER last_known_addr { host: "127.26.120.252" port: 36809 } attrs { promote: true } } }
I20260929 10:59:37.861214 1496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:37.861935 1496 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d: attempting to promote NON_VOTER 7e0f409104114e16aa509d93ea23e480 to VOTER
I20260929 10:59:37.862073 1496 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: false } } peers { permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 36809 } attrs { promote: false } }
I20260929 10:59:37.862488 1540 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [term 1 LEARNER]: Refusing update from remote peer 5c9a8ea9e35a43a99072b3ab64ca4b8d: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260929 10:59:37.862543 1354 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [term 1 FOLLOWER]: Refusing update from remote peer 5c9a8ea9e35a43a99072b3ab64ca4b8d: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260929 10:59:37.862715 2023 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 36809 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260929 10:59:37.862819 1997 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260929 10:59:37.863500 1496 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 7e0f409104114e16aa509d93ea23e480 (127.26.120.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: false } } peers { permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 36809 } attrs { promote: false } } }
I20260929 10:59:37.863598 1540 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 7e0f409104114e16aa509d93ea23e480 (127.26.120.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: false } } peers { permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 36809 } attrs { promote: false } } }
I20260929 10:59:37.864326 1496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: false } } peers { permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 36809 } attrs { promote: false } } }
I20260929 10:59:37.864346 2022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: false } } peers { permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 36809 } attrs { promote: false } } }
I20260929 10:59:37.864409 1496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:37.864421 2022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:37.864470 1496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: false } } peers { permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 36809 } attrs { promote: false } } }
I20260929 10:59:37.864442 1569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: false } } peers { permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 36809 } attrs { promote: false } } }
I20260929 10:59:37.864521 1496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:37.864531 1569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:59:37.864610 1354 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 7e0f409104114e16aa509d93ea23e480 (127.26.120.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: false } } peers { permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 36809 } attrs { promote: false } } }
I20260929 10:59:37.865351 2025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: false } } peers { permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 36809 } attrs { promote: false } } }
I20260929 10:59:37.865447 2025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:59:37.876206 2005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.873868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48724 (local address 127.26.120.194:44103)
0929 10:59:37.873924 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:37.873925 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:37.873932 (+ 7us) server_negotiation.cc:407] Connection header received
0929 10:59:37.873993 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:37.873994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:37.874010 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:37.874127 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:59:37.874338 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.874790 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.875475 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.875679 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.875726 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.875893 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:59:37.876000 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:59:37.876060 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:37.876098 (+ 38us) server_negotiation.cc:299] Negotiation successful
0929 10:59:37.876113 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 10:59:37.877810 1936 raft_consensus.cc:1261] T 780ac9a706e040fbabacc78a45de0ec1 P 83065b57befe412fbf130152240e8700 [term 1 FOLLOWER]: Refusing update from remote peer b586e21fea2f46c385b5f49e297018b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:59:37.877847 1663 raft_consensus.cc:1261] T 780ac9a706e040fbabacc78a45de0ec1 P deddd697abc04e13b689f98668eb1250 [term 1 FOLLOWER]: Refusing update from remote peer b586e21fea2f46c385b5f49e297018b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:59:37.878171 2020 consensus_queue.cc:1035] T 780ac9a706e040fbabacc78a45de0ec1 P b586e21fea2f46c385b5f49e297018b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83065b57befe412fbf130152240e8700" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:59:37.878321 2014 consensus_queue.cc:1035] T 780ac9a706e040fbabacc78a45de0ec1 P b586e21fea2f46c385b5f49e297018b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "deddd697abc04e13b689f98668eb1250" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:59:37.880591 2027 mvcc.cc:205] Tried to move back new op lower bound from 7334623550984912896 to 7334623550913744896. Current Snapshot: MvccSnapshot[applied={T|T < 7334623550984912896}]
I20260929 10:59:37.882851 2028 mvcc.cc:205] Tried to move back new op lower bound from 7334623550984912896 to 7334623550913744896. Current Snapshot: MvccSnapshot[applied={T|T < 7334623550984912896 or (T in {7334623550984912896})}]
I20260929 10:59:37.883417 747 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:59:37.883510 747 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:59:37.885121 747 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5c9a8ea9e35a43a99072b3ab64ca4b8d: Sending DeleteTablet for 3 replicas of tablet 780ac9a706e040fbabacc78a45de0ec1
I20260929 10:59:37.885437 1780 tablet_service.cc:1514] Processing DeleteTablet for tablet 780ac9a706e040fbabacc78a45de0ec1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:37 UTC) from {username='oryx', principal='oryx/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:48720
I20260929 10:59:37.885536 1916 tablet_service.cc:1514] Processing DeleteTablet for tablet 780ac9a706e040fbabacc78a45de0ec1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:37 UTC) from {username='oryx', principal='oryx/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:37540
I20260929 10:59:37.885519 1643 tablet_service.cc:1514] Processing DeleteTablet for tablet 780ac9a706e040fbabacc78a45de0ec1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:37 UTC) from {username='oryx', principal='oryx/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:44578
I20260929 10:59:37.885771 2038 tablet_replica.cc:330] T 780ac9a706e040fbabacc78a45de0ec1 P deddd697abc04e13b689f98668eb1250: stopping tablet replica
I20260929 10:59:37.885888 2038 raft_consensus.cc:2229] T 780ac9a706e040fbabacc78a45de0ec1 P deddd697abc04e13b689f98668eb1250 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:59:37.886003 2038 raft_consensus.cc:2258] T 780ac9a706e040fbabacc78a45de0ec1 P deddd697abc04e13b689f98668eb1250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:37.886745 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 715
I20260929 10:59:37.890223 2037 tablet_replica.cc:330] T 780ac9a706e040fbabacc78a45de0ec1 P b586e21fea2f46c385b5f49e297018b7: stopping tablet replica
I20260929 10:59:37.890334 2037 raft_consensus.cc:2229] T 780ac9a706e040fbabacc78a45de0ec1 P b586e21fea2f46c385b5f49e297018b7 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:59:37.890431 2037 raft_consensus.cc:2258] T 780ac9a706e040fbabacc78a45de0ec1 P b586e21fea2f46c385b5f49e297018b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:37.891386 2037 ts_tablet_manager.cc:1905] T 780ac9a706e040fbabacc78a45de0ec1 P b586e21fea2f46c385b5f49e297018b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:37.891650 2038 ts_tablet_manager.cc:1905] T 780ac9a706e040fbabacc78a45de0ec1 P deddd697abc04e13b689f98668eb1250: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:37.893132 2037 ts_tablet_manager.cc:1918] T 780ac9a706e040fbabacc78a45de0ec1 P b586e21fea2f46c385b5f49e297018b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:59:37.893260 2037 log.cc:1196] T 780ac9a706e040fbabacc78a45de0ec1 P b586e21fea2f46c385b5f49e297018b7: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-1/wal/wals/780ac9a706e040fbabacc78a45de0ec1
I20260929 10:59:37.893477 2038 ts_tablet_manager.cc:1918] T 780ac9a706e040fbabacc78a45de0ec1 P deddd697abc04e13b689f98668eb1250: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:59:37.893569 2038 log.cc:1196] T 780ac9a706e040fbabacc78a45de0ec1 P deddd697abc04e13b689f98668eb1250: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-0/wal/wals/780ac9a706e040fbabacc78a45de0ec1
I20260929 10:59:37.893714 2037 ts_tablet_manager.cc:1939] T 780ac9a706e040fbabacc78a45de0ec1 P b586e21fea2f46c385b5f49e297018b7: Deleting consensus metadata
I20260929 10:59:37.893862 2038 ts_tablet_manager.cc:1939] T 780ac9a706e040fbabacc78a45de0ec1 P deddd697abc04e13b689f98668eb1250: Deleting consensus metadata
W20260929 10:59:37.894243 1596 connection.cc:441] server connection from 127.0.0.1:44578 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44578 (request call id 1) could send its response
W20260929 10:59:37.895252 1734 connection.cc:441] server connection from 127.0.0.1:48720 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:48720 (request call id 1) could send its response
I20260929 10:59:37.898164 2039 tablet_replica.cc:330] T 780ac9a706e040fbabacc78a45de0ec1 P 83065b57befe412fbf130152240e8700: stopping tablet replica
I20260929 10:59:37.898293 2039 raft_consensus.cc:2229] T 780ac9a706e040fbabacc78a45de0ec1 P 83065b57befe412fbf130152240e8700 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:59:37.898455 2039 raft_consensus.cc:2258] T 780ac9a706e040fbabacc78a45de0ec1 P 83065b57befe412fbf130152240e8700 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:37.899016 2039 ts_tablet_manager.cc:1905] T 780ac9a706e040fbabacc78a45de0ec1 P 83065b57befe412fbf130152240e8700: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:37.899355 1716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.886273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48296 (local address 127.26.120.252:36809)
0929 10:59:37.886341 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:37.886343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:37.886351 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:59:37.886449 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:37.886450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:37.886459 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:37.886489 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:37.888086 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.888955 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.890280 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.890574 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.890628 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.895813 (+ 5185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:37.895814 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:37.895815 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:37.895823 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:37.897586 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.898001 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.898003 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.898003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.898085 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.898418 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.898419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.898420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.898555 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:37.898647 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:37.899174 (+ 527us) server_negotiation.cc:299] Negotiation successful
0929 10:59:37.899251 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 10:59:37.900705 2039 ts_tablet_manager.cc:1918] T 780ac9a706e040fbabacc78a45de0ec1 P 83065b57befe412fbf130152240e8700: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:59:37.900785 2039 log.cc:1196] T 780ac9a706e040fbabacc78a45de0ec1 P 83065b57befe412fbf130152240e8700: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-2/wal/wals/780ac9a706e040fbabacc78a45de0ec1
I20260929 10:59:37.901077 2039 ts_tablet_manager.cc:1939] T 780ac9a706e040fbabacc78a45de0ec1 P 83065b57befe412fbf130152240e8700: Deleting consensus metadata
W20260929 10:59:37.901456 1871 connection.cc:441] server connection from 127.0.0.1:37540 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37540 (request call id 1) could send its response
I20260929 10:59:37.903898 2047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.895724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48310 (local address 127.26.120.252:36809)
0929 10:59:37.895930 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:37.895932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:37.896174 (+ 242us) server_negotiation.cc:407] Connection header received
0929 10:59:37.896197 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:37.896198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:37.896209 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:37.896263 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:37.896822 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.897560 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.898718 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.899014 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.899079 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.900903 (+ 1824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:37.900905 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:37.900905 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:37.900912 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:37.902660 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.903108 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.903109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.903109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.903169 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.903340 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.903341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.903341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.903474 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:37.903587 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:37.903702 (+ 115us) server_negotiation.cc:299] Negotiation successful
0929 10:59:37.903797 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":52,"threads_started":1}
I20260929 10:59:37.903995 1395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:37.895553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43496 (local address 127.26.120.253:33873)
0929 10:59:37.895645 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:37.895647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:37.895888 (+ 241us) server_negotiation.cc:407] Connection header received
0929 10:59:37.895994 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:37.895995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:37.896007 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:37.896039 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:37.896729 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.897506 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.898713 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:37.899010 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:37.899053 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:37.900794 (+ 1741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:37.900804 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:37.900805 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:37.900813 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:37.902753 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.903173 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.903174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.903174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.903235 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:37.903416 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:37.903417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:37.903417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:37.903544 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:37.903625 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:37.903819 (+ 194us) server_negotiation.cc:299] Negotiation successful
0929 10:59:37.903889 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 10:59:38.881806 2049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:38.881470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34785 (local address 127.26.120.194:60863)
0929 10:59:38.881659 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:38.881749 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":119,"threads_started":1}
W20260929 10:59:38.882083 1847 heartbeater.cc:644] Failed to heartbeat to 127.26.120.254:34785 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.120.254:34785: connect: Connection refused (error 111)
I20260929 10:59:38.882520 2050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:38.882315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34785 (local address 127.26.120.195:46497)
0929 10:59:38.882415 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:38.882477 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":46,"threads_started":1}
W20260929 10:59:38.882688 1983 heartbeater.cc:644] Failed to heartbeat to 127.26.120.254:34785 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.120.254:34785: connect: Connection refused (error 111)
I20260929 10:59:38.883242 2051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:38.882961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34785 (local address 127.26.120.193:42669)
0929 10:59:38.883146 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:38.883205 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":83,"threads_started":1}
W20260929 10:59:38.883399 1710 heartbeater.cc:644] Failed to heartbeat to 127.26.120.254:34785 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.120.254:34785: connect: Connection refused (error 111)
I20260929 10:59:39.385026 2052 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c9a8ea9e35a43a99072b3ab64ca4b8d)
I20260929 10:59:39.385186 2052 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: false } } peers { permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 36809 } attrs { promote: false } }
I20260929 10:59:39.385474 2052 leader_election.cc:288] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c9a8ea9e35a43a99072b3ab64ca4b8d (127.26.120.254:34785), 7e0f409104114e16aa509d93ea23e480 (127.26.120.252:36809)
I20260929 10:59:39.385699 2053 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c9a8ea9e35a43a99072b3ab64ca4b8d)
I20260929 10:59:39.385833 2053 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: false } } peers { permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 36809 } attrs { promote: false } }
I20260929 10:59:39.385937 2054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:39.385603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34785 (local address 127.0.0.1:32820)
0929 10:59:39.385845 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:39.385892 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":56,"threads_started":1}
W20260929 10:59:39.386103 1329 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.120.254:34785: connect: Connection refused (error 111)
I20260929 10:59:39.386279 2053 leader_election.cc:288] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c9a8ea9e35a43a99072b3ab64ca4b8d (127.26.120.254:34785), 2db81c82699747b3bd5bef1172282f3b (127.26.120.253:33873)
I20260929 10:59:39.386453 2057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:39.386160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34785 (local address 127.0.0.1:32836)
0929 10:59:39.386291 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:39.386416 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":48,"threads_started":1}
W20260929 10:59:39.386623 1515 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.120.254:34785: connect: Connection refused (error 111)
I20260929 10:59:39.386945 2054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:39.386854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34785 (local address 127.0.0.1:32844)
0929 10:59:39.386892 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:39.386914 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260929 10:59:39.387024 1329 leader_election.cc:334] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5c9a8ea9e35a43a99072b3ab64ca4b8d (127.26.120.254:34785): Network error: Client connection negotiation failed: client connection to 127.26.120.254:34785: connect: Connection refused (error 111)
I20260929 10:59:39.387248 2059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:39.387157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34785 (local address 127.0.0.1:32852)
0929 10:59:39.387196 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:39.387221 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260929 10:59:39.387337 1515 leader_election.cc:334] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5c9a8ea9e35a43a99072b3ab64ca4b8d (127.26.120.254:34785): Network error: Client connection negotiation failed: client connection to 127.26.120.254:34785: connect: Connection refused (error 111)
I20260929 10:59:39.389350 2055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:39.385746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:36809 (local address 127.0.0.1:48312)
0929 10:59:39.386019 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:39.386031 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:39.386128 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:39.386284 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:39.386285 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:39.386287 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:39.386357 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:39.386359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:39.387375 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:39.387376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:39.388513 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:39.388515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:39.389105 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:39.389106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:39.389179 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:39.389185 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 10:59:39.389244 (+ 59us) client_negotiation.cc:240] Negotiation successful
0929 10:59:39.389293 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":3,"thread_start_us":49,"threads_started":1}
I20260929 10:59:39.389359 2056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:39.385876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48312 (local address 127.26.120.252:36809)
0929 10:59:39.385961 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:39.385963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:39.386115 (+ 152us) server_negotiation.cc:407] Connection header received
0929 10:59:39.386198 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:39.386199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:39.386213 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:39.386421 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:39.386458 (+ 37us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:39.387223 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:39.388605 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:39.388982 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:39.389078 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:39.389088 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:39.389258 (+ 170us) server_negotiation.cc:299] Negotiation successful
0929 10:59:39.389299 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66,"thread_start_us":40,"threads_started":1}
I20260929 10:59:39.389719 1540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2db81c82699747b3bd5bef1172282f3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7e0f409104114e16aa509d93ea23e480" is_pre_election: true
I20260929 10:59:39.389838 1540 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2db81c82699747b3bd5bef1172282f3b in term 1.
I20260929 10:59:39.390039 1329 leader_election.cc:302] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2db81c82699747b3bd5bef1172282f3b, 7e0f409104114e16aa509d93ea23e480; no voters: 5c9a8ea9e35a43a99072b3ab64ca4b8d
I20260929 10:59:39.390172 2052 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260929 10:59:39.390208 2057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:39.386464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.253:33873 (local address 127.0.0.1:43504)
0929 10:59:39.386850 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:39.386861 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:39.386917 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:39.387213 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:39.387215 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:39.387217 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:39.387263 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:39.387265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:39.388320 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:39.388321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:39.389139 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:39.389141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:39.389939 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:39.389940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:39.390010 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:39.390014 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:59:39.390101 (+ 87us) client_negotiation.cc:240] Negotiation successful
0929 10:59:39.390145 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":67,"threads_started":1}
I20260929 10:59:39.390233 2052 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5c9a8ea9e35a43a99072b3ab64ca4b8d)
I20260929 10:59:39.390268 2052 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [term 1 FOLLOWER]: Advancing to term 2
I20260929 10:59:39.390370 2060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:39.386605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43504 (local address 127.26.120.253:33873)
0929 10:59:39.386727 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:39.386728 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:39.386966 (+ 238us) server_negotiation.cc:407] Connection header received
0929 10:59:39.387106 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:39.387107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:39.387117 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:39.387182 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:39.387432 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:39.388221 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:39.389248 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:39.389833 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:39.389923 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:39.389930 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:39.390127 (+ 197us) server_negotiation.cc:299] Negotiation successful
0929 10:59:39.390174 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":76,"server-negotiator.queue_time_us":103,"thread_start_us":76,"threads_started":1}
I20260929 10:59:39.390558 1354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7e0f409104114e16aa509d93ea23e480" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2db81c82699747b3bd5bef1172282f3b" is_pre_election: true
I20260929 10:59:39.391033 2052 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: false } } peers { permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 36809 } attrs { promote: false } }
I20260929 10:59:39.391186 2052 leader_election.cc:288] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [CANDIDATE]: Term 2 election: Requested vote from peers 5c9a8ea9e35a43a99072b3ab64ca4b8d (127.26.120.254:34785), 7e0f409104114e16aa509d93ea23e480 (127.26.120.252:36809)
I20260929 10:59:39.391185 1354 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e0f409104114e16aa509d93ea23e480 in current term 2: Already voted for candidate 2db81c82699747b3bd5bef1172282f3b in this term.
I20260929 10:59:39.391417 1514 leader_election.cc:302] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e0f409104114e16aa509d93ea23e480; no voters: 2db81c82699747b3bd5bef1172282f3b, 5c9a8ea9e35a43a99072b3ab64ca4b8d
I20260929 10:59:39.391430 2055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:39.391289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34785 (local address 127.0.0.1:32854)
0929 10:59:39.391379 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:39.391408 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
I20260929 10:59:39.391598 1540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2db81c82699747b3bd5bef1172282f3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7e0f409104114e16aa509d93ea23e480"
I20260929 10:59:39.391597 2053 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [term 1 FOLLOWER]: Advancing to term 2
I20260929 10:59:39.391909 2055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:39.391830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34785 (local address 127.0.0.1:32866)
0929 10:59:39.391866 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:39.391887 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260929 10:59:39.391980 1329 leader_election.cc:334] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5c9a8ea9e35a43a99072b3ab64ca4b8d (127.26.120.254:34785): Network error: Client connection negotiation failed: client connection to 127.26.120.254:34785: connect: Connection refused (error 111)
I20260929 10:59:39.392393 2053 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 10:59:39.393090 1540 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2db81c82699747b3bd5bef1172282f3b in term 2.
I20260929 10:59:39.393246 1329 leader_election.cc:302] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2db81c82699747b3bd5bef1172282f3b, 7e0f409104114e16aa509d93ea23e480; no voters: 5c9a8ea9e35a43a99072b3ab64ca4b8d
I20260929 10:59:39.393378 2052 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [term 2 FOLLOWER]: Leader election won for term 2
I20260929 10:59:39.393522 2052 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [term 2 LEADER]: Becoming Leader. State: Replica: 2db81c82699747b3bd5bef1172282f3b, State: Running, Role: LEADER
I20260929 10:59:39.393617 2052 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: false } } peers { permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 36809 } attrs { promote: false } }
I20260929 10:59:39.393951 2062 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2db81c82699747b3bd5bef1172282f3b. Latest consensus state: current_term: 2 leader_uuid: "2db81c82699747b3bd5bef1172282f3b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: false } } peers { permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 36809 } attrs { promote: false } } }
I20260929 10:59:39.394043 2062 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:39.394255 2064 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:59:39.394676 2064 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:59:39.394788 2064 catalog_manager.cc:1259] Loaded cluster ID: 27ec5d70879346ffba3bed1fd19d6dff
I20260929 10:59:39.394826 2064 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:59:39.395001 2064 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:59:39.395092 2064 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b: Loaded TSK: 0
I20260929 10:59:39.395244 2064 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 10:59:39.438318 1344 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43496:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:59:39.439961 1344 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:59:39.441022 2055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:39.440879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34785 (local address 127.0.0.1:32876)
0929 10:59:39.440951 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:39.440988 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260929 10:59:39.441135 1540 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [term 2 FOLLOWER]: Refusing update from remote peer 2db81c82699747b3bd5bef1172282f3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260929 10:59:39.441351 2055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:39.441266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34785 (local address 127.0.0.1:32886)
0929 10:59:39.441298 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:39.441327 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260929 10:59:39.441385 2052 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 36809 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260929 10:59:39.441491 1329 consensus_peers.cc:487] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b -> Peer 5c9a8ea9e35a43a99072b3ab64ca4b8d (127.26.120.254:34785): Couldn't send request to peer 5c9a8ea9e35a43a99072b3ab64ca4b8d. Status: Network error: Client connection negotiation failed: client connection to 127.26.120.254:34785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260929 10:59:39.442034 2053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2db81c82699747b3bd5bef1172282f3b. Latest consensus state: current_term: 2 leader_uuid: "2db81c82699747b3bd5bef1172282f3b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: false } } peers { permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 36809 } attrs { promote: false } } }
I20260929 10:59:39.442180 2053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:59:39.442582 2062 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "2db81c82699747b3bd5bef1172282f3b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: false } } peers { permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 36809 } attrs { promote: false } } }
I20260929 10:59:39.442692 2062 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:39.442652 2066 mvcc.cc:205] Tried to move back new op lower bound from 7334623557388341248 to 7334623557196775424. Current Snapshot: MvccSnapshot[applied={T|T < 7334623557388341248}]
I20260929 10:59:39.442829 2053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "2db81c82699747b3bd5bef1172282f3b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "5c9a8ea9e35a43a99072b3ab64ca4b8d" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34785 } } peers { permanent_uuid: "2db81c82699747b3bd5bef1172282f3b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 33873 } attrs { promote: false } } peers { permanent_uuid: "7e0f409104114e16aa509d93ea23e480" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 36809 } attrs { promote: false } } }
I20260929 10:59:39.442922 2053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e0f409104114e16aa509d93ea23e480 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 10:59:39.448048 2054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:39.444787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:43039 (local address 127.0.0.1:37544)
0929 10:59:39.444833 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:39.444964 (+ 131us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:39.445008 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:39.445183 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:39.445184 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:39.445188 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:39.445248 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:39.445250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:39.446257 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:39.446258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:39.446998 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:39.447000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:39.447825 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:39.447826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:39.447893 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:39.447898 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 10:59:39.447927 (+ 29us) client_negotiation.cc:240] Negotiation successful
0929 10:59:39.447969 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260929 10:59:39.448313 2072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:39.444887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44590 (local address 127.26.120.193:42371)
0929 10:59:39.445045 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:39.445047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:39.445055 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:59:39.445100 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:39.445101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:39.445111 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:39.445334 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:39.445372 (+ 38us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:39.446150 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:39.447456 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:39.448016 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:39.448097 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:39.448107 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:39.448205 (+ 98us) server_negotiation.cc:299] Negotiation successful
0929 10:59:39.448244 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":53,"threads_started":1}
I20260929 10:59:39.448313 2055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:39.444689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:42371 (local address 127.0.0.1:44590)
0929 10:59:39.444718 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:39.444730 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:39.444769 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:39.445244 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:39.445245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:39.445246 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:39.445273 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:39.445274 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:39.446217 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:39.446217 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:39.447370 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:39.447372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:39.448088 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:39.448089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:39.448142 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:39.448145 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:59:39.448202 (+ 57us) client_negotiation.cc:240] Negotiation successful
0929 10:59:39.448241 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15,"mutex_wait_us":66}
I20260929 10:59:39.448378 2071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:39.444872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37544 (local address 127.26.120.195:43039)
0929 10:59:39.444990 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:39.444993 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:39.445003 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:59:39.445068 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:39.445069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:39.445079 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:39.445317 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:39.445354 (+ 37us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:39.446158 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:39.447125 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:39.447730 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:39.448234 (+ 504us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:39.448244 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:39.448262 (+ 18us) server_negotiation.cc:299] Negotiation successful
0929 10:59:39.448300 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":66,"threads_started":1}
I20260929 10:59:39.448549 1643 tablet_service.cc:1467] Processing CreateTablet for tablet 720e3b43a8a04aa4ac321030ceebaff5 (DEFAULT_TABLE table=test-table [id=2b30f58a1dfd4e699df231747fd0705b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:39.448683 1643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 720e3b43a8a04aa4ac321030ceebaff5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:39.448668 1916 tablet_service.cc:1467] Processing CreateTablet for tablet 720e3b43a8a04aa4ac321030ceebaff5 (DEFAULT_TABLE table=test-table [id=2b30f58a1dfd4e699df231747fd0705b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:39.448789 1916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 720e3b43a8a04aa4ac321030ceebaff5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:39.448808 2073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:39.444927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48734 (local address 127.26.120.194:44103)
0929 10:59:39.445033 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:39.445035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:39.445191 (+ 156us) server_negotiation.cc:407] Connection header received
0929 10:59:39.445220 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:39.445221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:39.445232 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:39.445346 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:39.445549 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:39.446032 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:39.447512 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:39.447896 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:39.447996 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:39.448005 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:39.448657 (+ 652us) server_negotiation.cc:299] Negotiation successful
0929 10:59:39.448712 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":52,"threads_started":1}
I20260929 10:59:39.448987 2070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:39.444821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:44103 (local address 127.0.0.1:48734)
0929 10:59:39.445095 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:39.445103 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:39.445144 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:39.445394 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:39.445394 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:39.445397 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:39.445466 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:39.445468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:39.446165 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:39.446165 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:39.447450 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:39.447452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:39.447992 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:39.447993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:39.448070 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:39.448077 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 10:59:39.448337 (+ 260us) client_negotiation.cc:240] Negotiation successful
0929 10:59:39.448912 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":30,"thread_start_us":80,"threads_started":1}
I20260929 10:59:39.449258 1780 tablet_service.cc:1467] Processing CreateTablet for tablet 720e3b43a8a04aa4ac321030ceebaff5 (DEFAULT_TABLE table=test-table [id=2b30f58a1dfd4e699df231747fd0705b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:39.449368 1780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 720e3b43a8a04aa4ac321030ceebaff5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:39.450129 2074 tablet_bootstrap.cc:492] T 720e3b43a8a04aa4ac321030ceebaff5 P deddd697abc04e13b689f98668eb1250: Bootstrap starting.
I20260929 10:59:39.450354 2075 tablet_bootstrap.cc:492] T 720e3b43a8a04aa4ac321030ceebaff5 P 83065b57befe412fbf130152240e8700: Bootstrap starting.
I20260929 10:59:39.450560 2074 tablet_bootstrap.cc:654] T 720e3b43a8a04aa4ac321030ceebaff5 P deddd697abc04e13b689f98668eb1250: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:39.450680 2075 tablet_bootstrap.cc:654] T 720e3b43a8a04aa4ac321030ceebaff5 P 83065b57befe412fbf130152240e8700: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:39.450847 2076 tablet_bootstrap.cc:492] T 720e3b43a8a04aa4ac321030ceebaff5 P b586e21fea2f46c385b5f49e297018b7: Bootstrap starting.
I20260929 10:59:39.451149 2074 tablet_bootstrap.cc:492] T 720e3b43a8a04aa4ac321030ceebaff5 P deddd697abc04e13b689f98668eb1250: No bootstrap required, opened a new log
I20260929 10:59:39.451203 2074 ts_tablet_manager.cc:1397] T 720e3b43a8a04aa4ac321030ceebaff5 P deddd697abc04e13b689f98668eb1250: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 10:59:39.451236 2076 tablet_bootstrap.cc:654] T 720e3b43a8a04aa4ac321030ceebaff5 P b586e21fea2f46c385b5f49e297018b7: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:39.451339 2074 raft_consensus.cc:348] T 720e3b43a8a04aa4ac321030ceebaff5 P deddd697abc04e13b689f98668eb1250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deddd697abc04e13b689f98668eb1250" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42371 } } peers { permanent_uuid: "b586e21fea2f46c385b5f49e297018b7" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 44103 } } peers { permanent_uuid: "83065b57befe412fbf130152240e8700" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43039 } }
I20260929 10:59:39.451362 2075 tablet_bootstrap.cc:492] T 720e3b43a8a04aa4ac321030ceebaff5 P 83065b57befe412fbf130152240e8700: No bootstrap required, opened a new log
I20260929 10:59:39.451387 2074 raft_consensus.cc:374] T 720e3b43a8a04aa4ac321030ceebaff5 P deddd697abc04e13b689f98668eb1250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:39.451400 2074 raft_consensus.cc:729] T 720e3b43a8a04aa4ac321030ceebaff5 P deddd697abc04e13b689f98668eb1250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: deddd697abc04e13b689f98668eb1250, State: Initialized, Role: FOLLOWER
I20260929 10:59:39.451434 2075 ts_tablet_manager.cc:1397] T 720e3b43a8a04aa4ac321030ceebaff5 P 83065b57befe412fbf130152240e8700: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:59:39.451467 2074 consensus_queue.cc:260] T 720e3b43a8a04aa4ac321030ceebaff5 P deddd697abc04e13b689f98668eb1250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deddd697abc04e13b689f98668eb1250" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42371 } } peers { permanent_uuid: "b586e21fea2f46c385b5f49e297018b7" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 44103 } } peers { permanent_uuid: "83065b57befe412fbf130152240e8700" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43039 } }
I20260929 10:59:39.451556 2075 raft_consensus.cc:348] T 720e3b43a8a04aa4ac321030ceebaff5 P 83065b57befe412fbf130152240e8700 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deddd697abc04e13b689f98668eb1250" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42371 } } peers { permanent_uuid: "b586e21fea2f46c385b5f49e297018b7" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 44103 } } peers { permanent_uuid: "83065b57befe412fbf130152240e8700" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43039 } }
I20260929 10:59:39.451647 2075 raft_consensus.cc:374] T 720e3b43a8a04aa4ac321030ceebaff5 P 83065b57befe412fbf130152240e8700 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:39.451634 2074 ts_tablet_manager.cc:1428] T 720e3b43a8a04aa4ac321030ceebaff5 P deddd697abc04e13b689f98668eb1250: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:39.451683 2075 raft_consensus.cc:729] T 720e3b43a8a04aa4ac321030ceebaff5 P 83065b57befe412fbf130152240e8700 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83065b57befe412fbf130152240e8700, State: Initialized, Role: FOLLOWER
I20260929 10:59:39.451743 2075 consensus_queue.cc:260] T 720e3b43a8a04aa4ac321030ceebaff5 P 83065b57befe412fbf130152240e8700 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deddd697abc04e13b689f98668eb1250" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42371 } } peers { permanent_uuid: "b586e21fea2f46c385b5f49e297018b7" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 44103 } } peers { permanent_uuid: "83065b57befe412fbf130152240e8700" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43039 } }
I20260929 10:59:39.451921 2075 ts_tablet_manager.cc:1428] T 720e3b43a8a04aa4ac321030ceebaff5 P 83065b57befe412fbf130152240e8700: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260929 10:59:39.452062 2082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:39.451822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34785 (local address 127.26.120.193:53327)
0929 10:59:39.451965 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:39.452012 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":94,"threads_started":1}
I20260929 10:59:39.452257 2083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:39.452055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34785 (local address 127.26.120.195:48297)
0929 10:59:39.452171 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:39.452205 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":86,"threads_started":1}
I20260929 10:59:39.452924 2082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:39.452831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34785 (local address 127.26.120.193:33515)
0929 10:59:39.452869 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:39.452900 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260929 10:59:39.453117 2076 tablet_bootstrap.cc:492] T 720e3b43a8a04aa4ac321030ceebaff5 P b586e21fea2f46c385b5f49e297018b7: No bootstrap required, opened a new log
I20260929 10:59:39.453122 1711 heartbeater.cc:503] Master 127.26.120.253:33873 was elected leader, sending a full tablet report...
I20260929 10:59:39.453167 2076 ts_tablet_manager.cc:1397] T 720e3b43a8a04aa4ac321030ceebaff5 P b586e21fea2f46c385b5f49e297018b7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:59:39.453284 2076 raft_consensus.cc:348] T 720e3b43a8a04aa4ac321030ceebaff5 P b586e21fea2f46c385b5f49e297018b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deddd697abc04e13b689f98668eb1250" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42371 } } peers { permanent_uuid: "b586e21fea2f46c385b5f49e297018b7" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 44103 } } peers { permanent_uuid: "83065b57befe412fbf130152240e8700" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43039 } }
I20260929 10:59:39.453356 2076 raft_consensus.cc:374] T 720e3b43a8a04aa4ac321030ceebaff5 P b586e21fea2f46c385b5f49e297018b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:39.453379 2076 raft_consensus.cc:729] T 720e3b43a8a04aa4ac321030ceebaff5 P b586e21fea2f46c385b5f49e297018b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b586e21fea2f46c385b5f49e297018b7, State: Initialized, Role: FOLLOWER
I20260929 10:59:39.453424 2076 consensus_queue.cc:260] T 720e3b43a8a04aa4ac321030ceebaff5 P b586e21fea2f46c385b5f49e297018b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deddd697abc04e13b689f98668eb1250" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42371 } } peers { permanent_uuid: "b586e21fea2f46c385b5f49e297018b7" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 44103 } } peers { permanent_uuid: "83065b57befe412fbf130152240e8700" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43039 } }
I20260929 10:59:39.453681 2076 ts_tablet_manager.cc:1428] T 720e3b43a8a04aa4ac321030ceebaff5 P b586e21fea2f46c385b5f49e297018b7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:39.453863 2083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:39.452831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34785 (local address 127.26.120.195:44891)
0929 10:59:39.453812 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:39.453845 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":967}
I20260929 10:59:39.454010 2085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:39.453769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34785 (local address 127.26.120.194:60639)
0929 10:59:39.453936 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:39.453975 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":76,"threads_started":1}
I20260929 10:59:39.454250 1848 heartbeater.cc:503] Master 127.26.120.253:33873 was elected leader, sending a full tablet report...
I20260929 10:59:39.454478 2085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:39.454354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34785 (local address 127.26.120.194:57471)
0929 10:59:39.454430 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:39.454457 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260929 10:59:39.530032 2081 raft_consensus.cc:482] T 720e3b43a8a04aa4ac321030ceebaff5 P 83065b57befe412fbf130152240e8700 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:59:39.530161 2081 raft_consensus.cc:504] T 720e3b43a8a04aa4ac321030ceebaff5 P 83065b57befe412fbf130152240e8700 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deddd697abc04e13b689f98668eb1250" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42371 } } peers { permanent_uuid: "b586e21fea2f46c385b5f49e297018b7" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 44103 } } peers { permanent_uuid: "83065b57befe412fbf130152240e8700" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43039 } }
I20260929 10:59:39.530536 2081 leader_election.cc:288] T 720e3b43a8a04aa4ac321030ceebaff5 P 83065b57befe412fbf130152240e8700 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers deddd697abc04e13b689f98668eb1250 (127.26.120.193:42371), b586e21fea2f46c385b5f49e297018b7 (127.26.120.194:44103)
I20260929 10:59:39.534320 2073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:39.530763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:53721 (local address 127.26.120.194:44103)
0929 10:59:39.530802 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:39.530804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:39.530979 (+ 175us) server_negotiation.cc:407] Connection header received
0929 10:59:39.531014 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:39.531014 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:39.531022 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:39.531065 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:39.531222 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:39.531914 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:39.533382 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:39.533951 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:39.534014 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:39.534023 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:39.534214 (+ 191us) server_negotiation.cc:299] Negotiation successful
0929 10:59:39.534252 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 10:59:39.534320 2072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:39.530730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:56059 (local address 127.26.120.193:42371)
0929 10:59:39.530774 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:39.530776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:39.530786 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:59:39.530869 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:39.530871 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:39.530880 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:39.530928 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:39.531135 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:39.531901 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:39.533383 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:39.533920 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:39.533991 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:39.534003 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:39.534215 (+ 212us) server_negotiation.cc:299] Negotiation successful
0929 10:59:39.534252 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 10:59:39.534334 2083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:39.530680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:42371 (local address 127.26.120.195:56059)
0929 10:59:39.530731 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:39.530743 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:39.530794 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:39.530937 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:39.530938 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:39.530941 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:39.531035 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:39.531037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:39.531990 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:39.531990 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:39.533291 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:39.533294 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:39.534047 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:39.534048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:39.534145 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:39.534148 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:59:39.534196 (+ 48us) client_negotiation.cc:240] Negotiation successful
0929 10:59:39.534245 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":76}
I20260929 10:59:39.534341 2086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:39.530686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:44103 (local address 127.26.120.195:53721)
0929 10:59:39.530905 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:39.530914 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:39.530966 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:39.531088 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:39.531089 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:39.531090 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:39.531155 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:39.531156 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:39.531989 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:39.531990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:39.533291 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:39.533294 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:39.534026 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:39.534027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:39.534131 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:39.534137 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 10:59:39.534196 (+ 59us) client_negotiation.cc:240] Negotiation successful
0929 10:59:39.534245 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":83,"thread_start_us":109,"threads_started":1}
I20260929 10:59:39.534538 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "720e3b43a8a04aa4ac321030ceebaff5" candidate_uuid: "83065b57befe412fbf130152240e8700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "deddd697abc04e13b689f98668eb1250" is_pre_election: true
I20260929 10:59:39.534616 1663 raft_consensus.cc:2454] T 720e3b43a8a04aa4ac321030ceebaff5 P deddd697abc04e13b689f98668eb1250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83065b57befe412fbf130152240e8700 in term 0.
I20260929 10:59:39.534797 1870 leader_election.cc:302] T 720e3b43a8a04aa4ac321030ceebaff5 P 83065b57befe412fbf130152240e8700 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83065b57befe412fbf130152240e8700, deddd697abc04e13b689f98668eb1250; no voters:
I20260929 10:59:39.534947 2081 raft_consensus.cc:2781] T 720e3b43a8a04aa4ac321030ceebaff5 P 83065b57befe412fbf130152240e8700 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:59:39.534989 2081 raft_consensus.cc:482] T 720e3b43a8a04aa4ac321030ceebaff5 P 83065b57befe412fbf130152240e8700 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:59:39.535014 2081 raft_consensus.cc:3037] T 720e3b43a8a04aa4ac321030ceebaff5 P 83065b57befe412fbf130152240e8700 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:39.534955 1800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "720e3b43a8a04aa4ac321030ceebaff5" candidate_uuid: "83065b57befe412fbf130152240e8700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b586e21fea2f46c385b5f49e297018b7" is_pre_election: true
I20260929 10:59:39.535106 1800 raft_consensus.cc:2454] T 720e3b43a8a04aa4ac321030ceebaff5 P b586e21fea2f46c385b5f49e297018b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83065b57befe412fbf130152240e8700 in term 0.
I20260929 10:59:39.535804 2081 raft_consensus.cc:504] T 720e3b43a8a04aa4ac321030ceebaff5 P 83065b57befe412fbf130152240e8700 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deddd697abc04e13b689f98668eb1250" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42371 } } peers { permanent_uuid: "b586e21fea2f46c385b5f49e297018b7" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 44103 } } peers { permanent_uuid: "83065b57befe412fbf130152240e8700" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43039 } }
I20260929 10:59:39.535944 2081 leader_election.cc:288] T 720e3b43a8a04aa4ac321030ceebaff5 P 83065b57befe412fbf130152240e8700 [CANDIDATE]: Term 1 election: Requested vote from peers deddd697abc04e13b689f98668eb1250 (127.26.120.193:42371), b586e21fea2f46c385b5f49e297018b7 (127.26.120.194:44103)
I20260929 10:59:39.536089 1663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "720e3b43a8a04aa4ac321030ceebaff5" candidate_uuid: "83065b57befe412fbf130152240e8700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "deddd697abc04e13b689f98668eb1250"
I20260929 10:59:39.536170 1663 raft_consensus.cc:3037] T 720e3b43a8a04aa4ac321030ceebaff5 P deddd697abc04e13b689f98668eb1250 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:39.536173 1800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "720e3b43a8a04aa4ac321030ceebaff5" candidate_uuid: "83065b57befe412fbf130152240e8700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b586e21fea2f46c385b5f49e297018b7"
I20260929 10:59:39.536232 1800 raft_consensus.cc:3037] T 720e3b43a8a04aa4ac321030ceebaff5 P b586e21fea2f46c385b5f49e297018b7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:39.536938 1663 raft_consensus.cc:2454] T 720e3b43a8a04aa4ac321030ceebaff5 P deddd697abc04e13b689f98668eb1250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83065b57befe412fbf130152240e8700 in term 1.
I20260929 10:59:39.537097 1870 leader_election.cc:302] T 720e3b43a8a04aa4ac321030ceebaff5 P 83065b57befe412fbf130152240e8700 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83065b57befe412fbf130152240e8700, deddd697abc04e13b689f98668eb1250; no voters:
I20260929 10:59:39.537120 1800 raft_consensus.cc:2454] T 720e3b43a8a04aa4ac321030ceebaff5 P b586e21fea2f46c385b5f49e297018b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83065b57befe412fbf130152240e8700 in term 1.
I20260929 10:59:39.537200 2081 raft_consensus.cc:2781] T 720e3b43a8a04aa4ac321030ceebaff5 P 83065b57befe412fbf130152240e8700 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:59:39.537377 2081 raft_consensus.cc:686] T 720e3b43a8a04aa4ac321030ceebaff5 P 83065b57befe412fbf130152240e8700 [term 1 LEADER]: Becoming Leader. State: Replica: 83065b57befe412fbf130152240e8700, State: Running, Role: LEADER
I20260929 10:59:39.537493 1984 heartbeater.cc:503] Master 127.26.120.253:33873 was elected leader, sending a full tablet report...
I20260929 10:59:39.537467 2081 consensus_queue.cc:237] T 720e3b43a8a04aa4ac321030ceebaff5 P 83065b57befe412fbf130152240e8700 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deddd697abc04e13b689f98668eb1250" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42371 } } peers { permanent_uuid: "b586e21fea2f46c385b5f49e297018b7" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 44103 } } peers { permanent_uuid: "83065b57befe412fbf130152240e8700" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43039 } }
I20260929 10:59:39.538121 2086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:39.537980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34785 (local address 127.26.120.195:38313)
0929 10:59:39.538015 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:39.538090 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260929 10:59:39.538408 1343 catalog_manager.cc:5495] T 720e3b43a8a04aa4ac321030ceebaff5 P 83065b57befe412fbf130152240e8700 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83065b57befe412fbf130152240e8700 (127.26.120.195). New cstate: current_term: 1 leader_uuid: "83065b57befe412fbf130152240e8700" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deddd697abc04e13b689f98668eb1250" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b586e21fea2f46c385b5f49e297018b7" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 44103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83065b57befe412fbf130152240e8700" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 43039 } health_report { overall_health: HEALTHY } } }
I20260929 10:59:39.563699 2071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:39.561250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37550 (local address 127.26.120.195:43039)
0929 10:59:39.561317 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:39.561320 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:39.561328 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:59:39.561369 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:39.561370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:39.561387 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:39.561468 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:59:39.561627 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:39.562160 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:39.562911 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:39.563117 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:39.563207 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:39.563367 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:59:39.563472 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:59:39.563567 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:39.563587 (+ 20us) server_negotiation.cc:299] Negotiation successful
0929 10:59:39.563609 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 10:59:39.564832 1663 raft_consensus.cc:1261] T 720e3b43a8a04aa4ac321030ceebaff5 P deddd697abc04e13b689f98668eb1250 [term 1 FOLLOWER]: Refusing update from remote peer 83065b57befe412fbf130152240e8700: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:59:39.565006 1800 raft_consensus.cc:1261] T 720e3b43a8a04aa4ac321030ceebaff5 P b586e21fea2f46c385b5f49e297018b7 [term 1 FOLLOWER]: Refusing update from remote peer 83065b57befe412fbf130152240e8700: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:59:39.565135 2081 consensus_queue.cc:1035] T 720e3b43a8a04aa4ac321030ceebaff5 P 83065b57befe412fbf130152240e8700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "deddd697abc04e13b689f98668eb1250" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:59:39.565303 2081 consensus_queue.cc:1035] T 720e3b43a8a04aa4ac321030ceebaff5 P 83065b57befe412fbf130152240e8700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b586e21fea2f46c385b5f49e297018b7" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 44103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:59:39.565979 2082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:39.565815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34785 (local address 127.26.120.193:54217)
0929 10:59:39.565911 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:39.565945 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260929 10:59:39.566335 2085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:39.566122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34785 (local address 127.26.120.194:34557)
0929 10:59:39.566268 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:39.566299 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
I20260929 10:59:39.567251 2092 mvcc.cc:205] Tried to move back new op lower bound from 7334623557895208960 to 7334623557786140672. Current Snapshot: MvccSnapshot[applied={T|T < 7334623557895208960}]
I20260929 10:59:39.567479 2086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:39.567304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34785 (local address 127.26.120.195:39269)
0929 10:59:39.567420 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:39.567448 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260929 10:59:39.567870 2082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:39.567635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34785 (local address 127.26.120.193:35233)
0929 10:59:39.567814 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:39.567845 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163}
I20260929 10:59:39.568145 2085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:39.568026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34785 (local address 127.26.120.194:59763)
0929 10:59:39.568088 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:39.568113 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260929 10:59:39.568567 2086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:39.568466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:34785 (local address 127.26.120.195:50937)
0929 10:59:39.568500 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:39.568529 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260929 10:59:39.569756 1344 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:59:39.569846 1344 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:59:39.571099 1344 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2db81c82699747b3bd5bef1172282f3b: Sending DeleteTablet for 3 replicas of tablet 720e3b43a8a04aa4ac321030ceebaff5
I20260929 10:59:39.571411 1643 tablet_service.cc:1514] Processing DeleteTablet for tablet 720e3b43a8a04aa4ac321030ceebaff5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:39 UTC) from {username='oryx', principal='oryx/127.26.120.253@KRBTEST.COM'} at 127.0.0.1:44590
I20260929 10:59:39.571434 1916 tablet_service.cc:1514] Processing DeleteTablet for tablet 720e3b43a8a04aa4ac321030ceebaff5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:39 UTC) from {username='oryx', principal='oryx/127.26.120.253@KRBTEST.COM'} at 127.0.0.1:37544
I20260929 10:59:39.571616 2101 tablet_replica.cc:330] T 720e3b43a8a04aa4ac321030ceebaff5 P deddd697abc04e13b689f98668eb1250: stopping tablet replica
I20260929 10:59:39.571683 2101 raft_consensus.cc:2229] T 720e3b43a8a04aa4ac321030ceebaff5 P deddd697abc04e13b689f98668eb1250 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:59:39.571736 2101 raft_consensus.cc:2258] T 720e3b43a8a04aa4ac321030ceebaff5 P deddd697abc04e13b689f98668eb1250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:39.571749 1780 tablet_service.cc:1514] Processing DeleteTablet for tablet 720e3b43a8a04aa4ac321030ceebaff5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:39 UTC) from {username='oryx', principal='oryx/127.26.120.253@KRBTEST.COM'} at 127.0.0.1:48734
I20260929 10:59:39.571846 2102 tablet_replica.cc:330] T 720e3b43a8a04aa4ac321030ceebaff5 P 83065b57befe412fbf130152240e8700: stopping tablet replica
I20260929 10:59:39.571893 2102 raft_consensus.cc:2229] T 720e3b43a8a04aa4ac321030ceebaff5 P 83065b57befe412fbf130152240e8700 [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:59:39.571983 2102 raft_consensus.cc:2258] T 720e3b43a8a04aa4ac321030ceebaff5 P 83065b57befe412fbf130152240e8700 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:39.572190 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 1491
I20260929 10:59:39.572292 2101 ts_tablet_manager.cc:1905] T 720e3b43a8a04aa4ac321030ceebaff5 P deddd697abc04e13b689f98668eb1250: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:39.572937 2102 ts_tablet_manager.cc:1905] T 720e3b43a8a04aa4ac321030ceebaff5 P 83065b57befe412fbf130152240e8700: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:39.574885 2102 ts_tablet_manager.cc:1918] T 720e3b43a8a04aa4ac321030ceebaff5 P 83065b57befe412fbf130152240e8700: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:59:39.574949 2102 log.cc:1196] T 720e3b43a8a04aa4ac321030ceebaff5 P 83065b57befe412fbf130152240e8700: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790679549986039-27107-0/minicluster-data/ts-2/wal/wals/720e3b43a8a04aa4ac321030ceebaff5
I20260929 10:59:39.575186 2102 ts_tablet_manager.cc:1939] T 720e3b43a8a04aa4ac321030ceebaff5 P 83065b57befe412fbf130152240e8700: Deleting consensus metadata
I20260929 10:59:39.575757 1329 catalog_manager.cc:4841] TS 83065b57befe412fbf130152240e8700 (127.26.120.195:43039): tablet 720e3b43a8a04aa4ac321030ceebaff5 (table test-table [id=2b30f58a1dfd4e699df231747fd0705b]) successfully deleted
I20260929 10:59:39.578565 2103 tablet_replica.cc:330] T 720e3b43a8a04aa4ac321030ceebaff5 P b586e21fea2f46c385b5f49e297018b7: stopping tablet replica
I20260929 10:59:39.578639 2103 raft_consensus.cc:2229] T 720e3b43a8a04aa4ac321030ceebaff5 P b586e21fea2f46c385b5f49e297018b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:59:39.578691 2103 raft_consensus.cc:2258] T 720e3b43a8a04aa4ac321030ceebaff5 P b586e21fea2f46c385b5f49e297018b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:39.579077 2103 ts_tablet_manager.cc:1905] T 720e3b43a8a04aa4ac321030ceebaff5 P b586e21fea2f46c385b5f49e297018b7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260929 10:59:39.579620 2070 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:39.579405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:42371 (local address 127.0.0.1:44602)
0929 10:59:39.579454 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:39.579466 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:39.579509 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:39.579569 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.193:42371: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":32}
W20260929 10:59:39.579778 1329 catalog_manager.cc:4584] TS deddd697abc04e13b689f98668eb1250 (127.26.120.193:42371): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 720e3b43a8a04aa4ac321030ceebaff5: Network error: Client connection negotiation failed: client connection to 127.26.120.193:42371: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260929 10:59:39.579808 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 1717
I20260929 10:59:39.587037 2070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:39.586895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:44103 (local address 127.0.0.1:48748)
0929 10:59:39.586953 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:39.586984 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.194:44103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260929 10:59:39.587026 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 1853
I20260929 10:59:39.591764 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 1211
I20260929 10:59:39.596259 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 1398
2026-09-29T10:59:39Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (4809 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:59:39 dist-test-slave-2lzq krb5kdc[2106](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:59:39 dist-test-slave-2lzq krb5kdc[2106](info): set up 2 sockets
Sep 29 10:59:39 dist-test-slave-2lzq krb5kdc[2106](info): commencing operation
krb5kdc: starting...
W20260929 10:59:40.646643 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 10:59:40 dist-test-slave-2lzq krb5kdc[2106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:59:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:59:40Z Disabled control of system clock
WARNING: no policy specified for oryx/127.26.120.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.120.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.120.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.254.keytab.
Entry for principal oryx/127.26.120.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.254.keytab.
I20260929 10:59:40.768471 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:43649
--webserver_interface=127.26.120.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.254.keytab
--principal=oryx/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:44339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:43649
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:40.847863 2119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:40.848067 2119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:40.848089 2119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:40.849187 2119 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:59:40.849224 2119 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 10:59:40.849238 2119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:40.849251 2119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:40.849264 2119 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:59:40.849277 2119 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:59:40.850744 2119 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:44339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:43649
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.254.keytab
--principal=oryx/127.26.120.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:43649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:40.850940 2119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:40.851109 2119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:40.856045 2124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:40.856132 2119 server_base.cc:1034] running on GCE node
W20260929 10:59:40.856041 2127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:40.856041 2125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:40.856498 2119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:40.856791 2119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:40.857967 2119 hybrid_clock.cc:648] HybridClock initialized: now 1790679580857952 us; error 27 us; skew 500 ppm
Sep 29 10:59:40 dist-test-slave-2lzq krb5kdc[2106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679580, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:40.860270 2119 init.cc:377] Logged in from keytab as oryx/127.26.120.254@KRBTEST.COM (short username oryx)
I20260929 10:59:40.860728 2119 webserver.cc:460] Webserver started at http://127.26.120.254:33183/ using document root <none> and password file <none>
I20260929 10:59:40.860966 2119 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:40.861045 2119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:40.861197 2119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:40.862291 2119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "5b33e54e771e4830afa648431a0880a2"
format_stamp: "Formatted at 2026-09-29 10:59:40 on dist-test-slave-2lzq"
server_key: "632fd29a59296a8a215245b1458d743b"
server_key_iv: "dc33ce58d50fd754034432f7e13ee327"
server_key_version: "encryptionkey@0"
I20260929 10:59:40.862653 2119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "5b33e54e771e4830afa648431a0880a2"
format_stamp: "Formatted at 2026-09-29 10:59:40 on dist-test-slave-2lzq"
server_key: "632fd29a59296a8a215245b1458d743b"
server_key_iv: "dc33ce58d50fd754034432f7e13ee327"
server_key_version: "encryptionkey@0"
I20260929 10:59:40.863938 2119 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 10:59:40.864647 2134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:40.864820 2119 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:59:40.864920 2119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "5b33e54e771e4830afa648431a0880a2"
format_stamp: "Formatted at 2026-09-29 10:59:40 on dist-test-slave-2lzq"
server_key: "632fd29a59296a8a215245b1458d743b"
server_key_iv: "dc33ce58d50fd754034432f7e13ee327"
server_key_version: "encryptionkey@0"
I20260929 10:59:40.865005 2119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:40.879845 2119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:40.881611 2119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:40.881780 2119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:40.886438 2119 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:43649
I20260929 10:59:40.886528 2196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:43649 every 8 connection(s)
I20260929 10:59:40.886868 2119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 10:59:40.887519 2197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:40.889609 2197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b33e54e771e4830afa648431a0880a2: Bootstrap starting.
I20260929 10:59:40.890232 2197 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b33e54e771e4830afa648431a0880a2: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:40.890477 2197 log.cc:824] T 00000000000000000000000000000000 P 5b33e54e771e4830afa648431a0880a2: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:40.891109 2197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b33e54e771e4830afa648431a0880a2: No bootstrap required, opened a new log
I20260929 10:59:40.893138 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 2119
I20260929 10:59:40.893224 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 10:59:40.893114 2197 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b33e54e771e4830afa648431a0880a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b33e54e771e4830afa648431a0880a2" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43649 } }
I20260929 10:59:40.893291 2197 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b33e54e771e4830afa648431a0880a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:40.893330 2197 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b33e54e771e4830afa648431a0880a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b33e54e771e4830afa648431a0880a2, State: Initialized, Role: FOLLOWER
I20260929 10:59:40.893404 27107 external_mini_cluster.cc:1428] Setting key 4905f8b0730340a00b786f9b6fa75e11
I20260929 10:59:40.893517 2197 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b33e54e771e4830afa648431a0880a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b33e54e771e4830afa648431a0880a2" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43649 } }
I20260929 10:59:40.893610 2197 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b33e54e771e4830afa648431a0880a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:59:40.893635 2197 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b33e54e771e4830afa648431a0880a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:59:40.893708 2197 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b33e54e771e4830afa648431a0880a2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:40.894361 2197 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b33e54e771e4830afa648431a0880a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b33e54e771e4830afa648431a0880a2" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43649 } }
I20260929 10:59:40.894483 2197 leader_election.cc:302] T 00000000000000000000000000000000 P 5b33e54e771e4830afa648431a0880a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b33e54e771e4830afa648431a0880a2; no voters:
I20260929 10:59:40.894663 2197 leader_election.cc:288] T 00000000000000000000000000000000 P 5b33e54e771e4830afa648431a0880a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:59:40.894695 2202 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b33e54e771e4830afa648431a0880a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:59:40.894794 2202 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b33e54e771e4830afa648431a0880a2 [term 1 LEADER]: Becoming Leader. State: Replica: 5b33e54e771e4830afa648431a0880a2, State: Running, Role: LEADER
I20260929 10:59:40.894872 2202 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b33e54e771e4830afa648431a0880a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b33e54e771e4830afa648431a0880a2" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43649 } }
I20260929 10:59:40.894914 2197 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b33e54e771e4830afa648431a0880a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:59:40.895182 2202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b33e54e771e4830afa648431a0880a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b33e54e771e4830afa648431a0880a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b33e54e771e4830afa648431a0880a2" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43649 } } }
I20260929 10:59:40.895248 2202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b33e54e771e4830afa648431a0880a2 [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:40.895362 2202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b33e54e771e4830afa648431a0880a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b33e54e771e4830afa648431a0880a2. Latest consensus state: current_term: 1 leader_uuid: "5b33e54e771e4830afa648431a0880a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b33e54e771e4830afa648431a0880a2" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43649 } } }
I20260929 10:59:40.895430 2202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b33e54e771e4830afa648431a0880a2 [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:40.895581 2207 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:59:40.896102 2207 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:59:40.897733 2207 catalog_manager.cc:1347] Generated new cluster ID: 473848061a214d748b6c79e4d2f13b70
I20260929 10:59:40.897785 2207 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 10:59:40 dist-test-slave-2lzq krb5kdc[2106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM
I20260929 10:59:40.904075 2201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:40.893899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59614 (local address 127.26.120.254:43649)
0929 10:59:40.894080 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:40.894090 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:40.894128 (+ 38us) server_negotiation.cc:407] Connection header received
0929 10:59:40.894255 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:40.894261 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:40.894303 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:40.894445 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:40.895251 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:40.896686 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:40.897745 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:40.898299 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:40.898425 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:40.900755 (+ 2330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:40.900758 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:40.900759 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:40.900770 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:40.902519 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:40.902911 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:40.902914 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:40.902915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:40.903017 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:40.903238 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:40.903239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:40.903239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:40.903471 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:40.903670 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:40.903751 (+ 81us) server_negotiation.cc:299] Negotiation successful
0929 10:59:40.903890 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":77,"threads_started":1}
I20260929 10:59:40.908614 2207 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:59:40.909185 2207 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:59:40.915840 2207 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b33e54e771e4830afa648431a0880a2: Generated new TSK 0
I20260929 10:59:40.916033 2207 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.26.120.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.120.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.120.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.193.keytab.
Entry for principal oryx/127.26.120.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.193.keytab.
I20260929 10:59:40.948884 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.193:0
--local_ip_for_outbound_sockets=127.26.120.193
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:43649
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.193.keytab
--principal=oryx/127.26.120.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:44339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:41.044159 2223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:41.044355 2223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:41.044380 2223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:41.045554 2223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:41.045598 2223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:41.045619 2223 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 10:59:41.045668 2223 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.193
I20260929 10:59:41.047211 2223 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:44339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.193.keytab
--principal=oryx/127.26.120.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.120.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.120.254:43649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2223
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.193
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:41.047472 2223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:41.047725 2223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:41.053128 2228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:41.053177 2229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:41.053131 2223 server_base.cc:1034] running on GCE node
W20260929 10:59:41.053159 2231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:41.053653 2223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:41.053951 2223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:41.055147 2223 hybrid_clock.cc:648] HybridClock initialized: now 1790679581055132 us; error 28 us; skew 500 ppm
Sep 29 10:59:41 dist-test-slave-2lzq krb5kdc[2106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679581, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:41.057585 2223 init.cc:377] Logged in from keytab as oryx/127.26.120.193@KRBTEST.COM (short username oryx)
I20260929 10:59:41.057994 2223 webserver.cc:460] Webserver started at http://127.26.120.193:43523/ using document root <none> and password file <none>
I20260929 10:59:41.058257 2223 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:41.058328 2223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:41.058476 2223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:41.059494 2223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-0/data/instance:
uuid: "297c51b2364b4beb90b2d1f5282ac70a"
format_stamp: "Formatted at 2026-09-29 10:59:41 on dist-test-slave-2lzq"
server_key: "af127c2c0e88f7e51cac4355c9da9686"
server_key_iv: "12f5e27c69e6be6d1f747772f7eede81"
server_key_version: "encryptionkey@0"
I20260929 10:59:41.059847 2223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance:
uuid: "297c51b2364b4beb90b2d1f5282ac70a"
format_stamp: "Formatted at 2026-09-29 10:59:41 on dist-test-slave-2lzq"
server_key: "af127c2c0e88f7e51cac4355c9da9686"
server_key_iv: "12f5e27c69e6be6d1f747772f7eede81"
server_key_version: "encryptionkey@0"
I20260929 10:59:41.061188 2223 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:59:41.061884 2238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:41.062119 2223 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:59:41.062223 2223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-0/wal
uuid: "297c51b2364b4beb90b2d1f5282ac70a"
format_stamp: "Formatted at 2026-09-29 10:59:41 on dist-test-slave-2lzq"
server_key: "af127c2c0e88f7e51cac4355c9da9686"
server_key_iv: "12f5e27c69e6be6d1f747772f7eede81"
server_key_version: "encryptionkey@0"
I20260929 10:59:41.062309 2223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:41.076176 2223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:41.078141 2223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:41.078294 2223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:41.078791 2223 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:41.078862 2223 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:41.078902 2223 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:41.078928 2223 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:41.085378 2223 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.193:46459
I20260929 10:59:41.085548 2353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.193:46459 every 8 connection(s)
I20260929 10:59:41.085758 2223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
Sep 29 10:59:41 dist-test-slave-2lzq krb5kdc[2106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679581, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM
I20260929 10:59:41.089254 2201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.079281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:38131 (local address 127.26.120.254:43649)
0929 10:59:41.079367 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:41.079370 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:41.079556 (+ 186us) server_negotiation.cc:407] Connection header received
0929 10:59:41.080126 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:41.080127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:41.080144 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:41.080224 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:41.081111 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.081966 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.083611 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.083966 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.084041 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.086707 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:41.086708 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:41.086709 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:41.086717 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:41.088173 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:41.088538 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:41.088539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:41.088539 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:41.088588 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:41.088784 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:41.088784 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:41.088784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:41.088881 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:41.088929 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:41.089126 (+ 197us) server_negotiation.cc:299] Negotiation successful
0929 10:59:41.089192 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 10:59:41.089422 2247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.079262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43649 (local address 127.26.120.193:38131)
0929 10:59:41.079434 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:41.079473 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:41.079976 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:41.080324 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:41.080328 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:41.080711 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:41.080998 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.081005 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.082090 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.082091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.083499 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.083502 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.084070 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.084071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.084224 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.084836 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:41.084846 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:41.086621 (+ 1775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:41.088248 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:41.088253 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:41.088255 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:41.088459 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:41.088643 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:41.088643 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:41.088644 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:41.088732 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:41.088920 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:41.088922 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:41.089026 (+ 104us) client_negotiation.cc:769] Sending connection context
0929 10:59:41.089115 (+ 89us) client_negotiation.cc:240] Negotiation successful
0929 10:59:41.089230 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":104,"threads_started":1}
I20260929 10:59:41.089993 2354 heartbeater.cc:347] Connected to a master server at 127.26.120.254:43649
I20260929 10:59:41.090124 2354 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:41.090296 2354 heartbeater.cc:511] Master 127.26.120.254:43649 requested a full tablet report, sending...
I20260929 10:59:41.090802 2150 ts_manager.cc:194] Registered new tserver with Master: 297c51b2364b4beb90b2d1f5282ac70a (127.26.120.193:46459)
I20260929 10:59:41.091538 2150 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.120.193@KRBTEST.COM'} at 127.26.120.193:38131
I20260929 10:59:41.094269 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 2223
I20260929 10:59:41.094589 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance
I20260929 10:59:41.094836 27107 external_mini_cluster.cc:1428] Setting key 8538560624a2ddcf3686697fe3f0bcac
I20260929 10:59:41.096789 2201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.091156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:60195 (local address 127.26.120.254:43649)
0929 10:59:41.091193 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:41.091195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:41.091326 (+ 131us) server_negotiation.cc:407] Connection header received
0929 10:59:41.091400 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:41.091401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:41.091411 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:41.091454 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:41.091816 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.092292 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.093002 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.093224 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.093314 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.094119 (+ 805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:41.094120 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:41.094120 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:41.094128 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:41.095602 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:41.096006 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:41.096007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:41.096007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:41.096069 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:41.096272 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:41.096273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:41.096273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:41.096364 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:41.096448 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:41.096623 (+ 175us) server_negotiation.cc:299] Negotiation successful
0929 10:59:41.096701 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 10:59:41.096796 2360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.091096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43649 (local address 127.26.120.193:60195)
0929 10:59:41.091257 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:41.091266 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:41.091341 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:41.091467 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:41.091468 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:41.091693 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:41.091758 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.091760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.092391 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.092392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.092913 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.092915 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.093317 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.093318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.093389 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.093734 (+ 345us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:41.093738 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:41.093993 (+ 255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:41.095704 (+ 1711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:41.095705 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:41.095705 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:41.095933 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:41.096135 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:41.096136 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:41.096136 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:41.096205 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:41.096456 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:41.096457 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:41.096503 (+ 46us) client_negotiation.cc:769] Sending connection context
0929 10:59:41.096599 (+ 96us) client_negotiation.cc:240] Negotiation successful
0929 10:59:41.096701 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":78,"threads_started":1}
WARNING: no policy specified for oryx/127.26.120.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.120.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.120.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.194.keytab.
Entry for principal oryx/127.26.120.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.194.keytab.
I20260929 10:59:41.120889 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.194:0
--local_ip_for_outbound_sockets=127.26.120.194
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:43649
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.194.keytab
--principal=oryx/127.26.120.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:44339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:41.201478 2363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:41.201716 2363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:41.201756 2363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:41.203342 2363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:41.203408 2363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:41.203436 2363 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 10:59:41.203460 2363 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.194
I20260929 10:59:41.205669 2363 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:44339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.194.keytab
--principal=oryx/127.26.120.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.120.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.120.254:43649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.194
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:41.205874 2363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:41.206103 2363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:41.211267 2371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:41.211243 2369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:41.211452 2368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:41.211270 2363 server_base.cc:1034] running on GCE node
I20260929 10:59:41.211725 2363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:41.211968 2363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:41.213151 2363 hybrid_clock.cc:648] HybridClock initialized: now 1790679581213146 us; error 35 us; skew 500 ppm
Sep 29 10:59:41 dist-test-slave-2lzq krb5kdc[2106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679581, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:41.215728 2363 init.cc:377] Logged in from keytab as oryx/127.26.120.194@KRBTEST.COM (short username oryx)
I20260929 10:59:41.216176 2363 webserver.cc:460] Webserver started at http://127.26.120.194:43653/ using document root <none> and password file <none>
I20260929 10:59:41.216446 2363 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:41.216527 2363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:41.216678 2363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:41.217767 2363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-1/data/instance:
uuid: "cff46e36bd9e400087ced7da2dbf9f0e"
format_stamp: "Formatted at 2026-09-29 10:59:41 on dist-test-slave-2lzq"
server_key: "a1bcd3b1dc82dd3f02e5e51d757de11e"
server_key_iv: "a5e21bfe61e5ae99c81dcb15464571db"
server_key_version: "encryptionkey@0"
I20260929 10:59:41.218200 2363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance:
uuid: "cff46e36bd9e400087ced7da2dbf9f0e"
format_stamp: "Formatted at 2026-09-29 10:59:41 on dist-test-slave-2lzq"
server_key: "a1bcd3b1dc82dd3f02e5e51d757de11e"
server_key_iv: "a5e21bfe61e5ae99c81dcb15464571db"
server_key_version: "encryptionkey@0"
I20260929 10:59:41.219547 2363 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:59:41.220234 2378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:41.220404 2363 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:59:41.220522 2363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-1/wal
uuid: "cff46e36bd9e400087ced7da2dbf9f0e"
format_stamp: "Formatted at 2026-09-29 10:59:41 on dist-test-slave-2lzq"
server_key: "a1bcd3b1dc82dd3f02e5e51d757de11e"
server_key_iv: "a5e21bfe61e5ae99c81dcb15464571db"
server_key_version: "encryptionkey@0"
I20260929 10:59:41.220599 2363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:41.230955 2363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:41.233248 2363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:41.233413 2363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:41.233934 2363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:41.234001 2363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:41.234038 2363 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:41.234115 2363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:41.241146 2363 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.194:45525
I20260929 10:59:41.241226 2493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.194:45525 every 8 connection(s)
I20260929 10:59:41.241555 2363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
Sep 29 10:59:41 dist-test-slave-2lzq krb5kdc[2106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679581, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM
I20260929 10:59:41.244867 2201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.234466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:41675 (local address 127.26.120.254:43649)
0929 10:59:41.234530 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:41.234533 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:41.234753 (+ 220us) server_negotiation.cc:407] Connection header received
0929 10:59:41.235285 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:41.235286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:41.235299 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:41.235371 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:41.236191 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.237019 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.238923 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.239227 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.239291 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.242253 (+ 2962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:41.242255 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:41.242255 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:41.242262 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:41.243565 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:41.244066 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:41.244067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:41.244067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:41.244120 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:41.244341 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:41.244341 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:41.244342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:41.244448 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:41.244500 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:41.244739 (+ 239us) server_negotiation.cc:299] Negotiation successful
0929 10:59:41.244802 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 10:59:41.245083 2387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.234457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43649 (local address 127.26.120.194:41675)
0929 10:59:41.234603 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:41.234637 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:41.235166 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:41.235445 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:41.235449 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:41.235805 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:41.236091 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.236097 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.237135 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.237136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.238806 (+ 1670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.238810 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.239336 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.239336 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.239503 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.240221 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:41.240232 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:41.242155 (+ 1923us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:41.243646 (+ 1491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:41.243652 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:41.243654 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:41.243981 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:41.244182 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:41.244183 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:41.244183 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:41.244283 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:41.244506 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:41.244511 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:41.244624 (+ 113us) client_negotiation.cc:769] Sending connection context
0929 10:59:41.244706 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 10:59:41.244827 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":71,"threads_started":1}
I20260929 10:59:41.245653 2494 heartbeater.cc:347] Connected to a master server at 127.26.120.254:43649
I20260929 10:59:41.245751 2494 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:41.245759 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 2363
I20260929 10:59:41.245857 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance
I20260929 10:59:41.245946 2494 heartbeater.cc:511] Master 127.26.120.254:43649 requested a full tablet report, sending...
I20260929 10:59:41.246145 27107 external_mini_cluster.cc:1428] Setting key 8b96f99bf6a8f71528cfcf375f57cb34
I20260929 10:59:41.246438 2150 ts_manager.cc:194] Registered new tserver with Master: cff46e36bd9e400087ced7da2dbf9f0e (127.26.120.194:45525)
I20260929 10:59:41.246902 2150 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.120.194@KRBTEST.COM'} at 127.26.120.194:41675
I20260929 10:59:41.253190 2201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.246850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:57397 (local address 127.26.120.254:43649)
0929 10:59:41.246890 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:41.246891 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:41.247037 (+ 146us) server_negotiation.cc:407] Connection header received
0929 10:59:41.247106 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:41.247107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:41.247119 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:41.247173 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:41.247872 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.248500 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.249244 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.249430 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.249503 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.250507 (+ 1004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:41.250508 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:41.250508 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:41.250516 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:41.251927 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:41.252285 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:41.252286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:41.252287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:41.252344 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:41.252552 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:41.252553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:41.252553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:41.252696 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:41.252772 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:41.253013 (+ 241us) server_negotiation.cc:299] Negotiation successful
0929 10:59:41.253100 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 10:59:41.253201 2500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.246857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43649 (local address 127.26.120.194:57397)
0929 10:59:41.246976 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:41.246987 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:41.247046 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:41.247176 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:41.247177 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:41.247737 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:41.247784 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.247786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.248611 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.248612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.249142 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.249144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.249513 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.249514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.249579 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.250032 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:41.250037 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:41.250413 (+ 376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:41.252032 (+ 1619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:41.252033 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:41.252033 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:41.252209 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:41.252410 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:41.252411 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:41.252412 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:41.252470 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:41.252775 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:41.252776 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:41.252822 (+ 46us) client_negotiation.cc:769] Sending connection context
0929 10:59:41.252982 (+ 160us) client_negotiation.cc:240] Negotiation successful
0929 10:59:41.253108 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":70,"threads_started":1}
WARNING: no policy specified for oryx/127.26.120.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.120.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.120.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.195.keytab.
Entry for principal oryx/127.26.120.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.195.keytab.
I20260929 10:59:41.272300 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.195:0
--local_ip_for_outbound_sockets=127.26.120.195
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:43649
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.195.keytab
--principal=oryx/127.26.120.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:44339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:41.351764 2503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:41.352018 2503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:41.352059 2503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:41.353229 2503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:41.353302 2503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:41.353338 2503 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 10:59:41.353375 2503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.195
I20260929 10:59:41.354887 2503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:44339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.195.keytab
--principal=oryx/127.26.120.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.120.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.120.254:43649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.195
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:41.355150 2503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:41.355382 2503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:41.360321 2509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:41.360306 2511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:41.360308 2508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:41.360567 2503 server_base.cc:1034] running on GCE node
I20260929 10:59:41.360785 2503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:41.361027 2503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:41.362195 2503 hybrid_clock.cc:648] HybridClock initialized: now 1790679581362190 us; error 37 us; skew 500 ppm
Sep 29 10:59:41 dist-test-slave-2lzq krb5kdc[2106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679581, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:41.364782 2503 init.cc:377] Logged in from keytab as oryx/127.26.120.195@KRBTEST.COM (short username oryx)
I20260929 10:59:41.365304 2503 webserver.cc:460] Webserver started at http://127.26.120.195:42437/ using document root <none> and password file <none>
I20260929 10:59:41.365545 2503 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:41.365617 2503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:41.365753 2503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:41.367013 2503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-2/data/instance:
uuid: "c32b27909a9949f48f3ea22489973ace"
format_stamp: "Formatted at 2026-09-29 10:59:41 on dist-test-slave-2lzq"
server_key: "249dae837c5ff6000397d89e645fbfaf"
server_key_iv: "95bfde0d6e9959e78dd952d0356afac8"
server_key_version: "encryptionkey@0"
I20260929 10:59:41.367398 2503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance:
uuid: "c32b27909a9949f48f3ea22489973ace"
format_stamp: "Formatted at 2026-09-29 10:59:41 on dist-test-slave-2lzq"
server_key: "249dae837c5ff6000397d89e645fbfaf"
server_key_iv: "95bfde0d6e9959e78dd952d0356afac8"
server_key_version: "encryptionkey@0"
I20260929 10:59:41.368757 2503 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 10:59:41.369490 2518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:41.369673 2503 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:59:41.369773 2503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-2/wal
uuid: "c32b27909a9949f48f3ea22489973ace"
format_stamp: "Formatted at 2026-09-29 10:59:41 on dist-test-slave-2lzq"
server_key: "249dae837c5ff6000397d89e645fbfaf"
server_key_iv: "95bfde0d6e9959e78dd952d0356afac8"
server_key_version: "encryptionkey@0"
I20260929 10:59:41.369877 2503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:41.381835 2503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:41.384184 2503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:41.384356 2503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:41.384864 2503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:41.384930 2503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:41.384964 2503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:41.384994 2503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 10:59:41 dist-test-slave-2lzq krb5kdc[2106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679581, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM
I20260929 10:59:41.392392 2503 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.195:35405
I20260929 10:59:41.392467 2633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.195:35405 every 8 connection(s)
I20260929 10:59:41.392807 2503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
I20260929 10:59:41.395579 2201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.385325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:55317 (local address 127.26.120.254:43649)
0929 10:59:41.385402 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:41.385405 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:41.385600 (+ 195us) server_negotiation.cc:407] Connection header received
0929 10:59:41.386064 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:41.386066 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:41.386078 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:41.386144 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:41.386882 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.387629 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.389366 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.389727 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.389800 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.392537 (+ 2737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:41.392564 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:41.392565 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:41.392574 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:41.394019 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:41.394515 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:41.394517 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:41.394517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:41.394579 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:41.394857 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:41.394858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:41.394858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:41.394986 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:41.395097 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:41.395403 (+ 306us) server_negotiation.cc:299] Negotiation successful
0929 10:59:41.395487 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 10:59:41.395771 2527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.385303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43649 (local address 127.26.120.195:55317)
0929 10:59:41.385439 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:41.385478 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:41.385950 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:41.386199 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:41.386203 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:41.386425 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:41.386766 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.386772 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.387730 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.387731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.389226 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.389230 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.389844 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.389845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.390013 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.390581 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:41.390595 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:41.392429 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:41.394180 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:41.394187 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:41.394189 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:41.394415 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:41.394658 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:41.394660 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:41.394660 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:41.394782 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:41.395114 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:41.395121 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:41.395261 (+ 140us) client_negotiation.cc:769] Sending connection context
0929 10:59:41.395379 (+ 118us) client_negotiation.cc:240] Negotiation successful
0929 10:59:41.395536 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":74,"threads_started":1}
I20260929 10:59:41.396229 2634 heartbeater.cc:347] Connected to a master server at 127.26.120.254:43649
I20260929 10:59:41.396348 2634 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:41.396529 2634 heartbeater.cc:511] Master 127.26.120.254:43649 requested a full tablet report, sending...
I20260929 10:59:41.396929 2151 ts_manager.cc:194] Registered new tserver with Master: c32b27909a9949f48f3ea22489973ace (127.26.120.195:35405)
I20260929 10:59:41.396972 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 2503
I20260929 10:59:41.397063 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance
I20260929 10:59:41.397264 27107 external_mini_cluster.cc:1428] Setting key 0eb784a95675dc2a29bdf2b44e759585
I20260929 10:59:41.397459 2151 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.120.195@KRBTEST.COM'} at 127.26.120.195:55317
I20260929 10:59:41.398250 27107 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 10:59:41.403730 2640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.397252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43649 (local address 127.26.120.195:59221)
0929 10:59:41.397377 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:41.397389 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:41.397445 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:41.397745 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:41.397746 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:41.397993 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:41.398102 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.398104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.399001 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.399013 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.399516 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.399518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.400008 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.400009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.400101 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.400490 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:41.400495 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:41.400892 (+ 397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:41.402739 (+ 1847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:41.402740 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:41.402740 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:41.402958 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:41.403183 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:41.403183 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:41.403183 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:41.403236 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:41.403498 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:41.403499 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:41.403536 (+ 37us) client_negotiation.cc:769] Sending connection context
0929 10:59:41.403586 (+ 50us) client_negotiation.cc:240] Negotiation successful
0929 10:59:41.403660 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":76,"threads_started":1}
I20260929 10:59:41.403789 2201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.397579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:59221 (local address 127.26.120.254:43649)
0929 10:59:41.397640 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:41.397642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:41.397651 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:59:41.397669 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:41.397670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:41.397678 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:41.397742 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:41.398194 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.398898 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.399603 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.399923 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.399994 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.400983 (+ 989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:41.400984 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:41.400985 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:41.400990 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:41.402651 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:41.403045 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:41.403047 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:41.403047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:41.403111 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:41.403297 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:41.403298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:41.403298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:41.403433 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:41.403508 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:41.403614 (+ 106us) server_negotiation.cc:299] Negotiation successful
0929 10:59:41.403691 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
Sep 29 10:59:41 dist-test-slave-2lzq krb5kdc[2106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679581, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 10:59:41 dist-test-slave-2lzq krb5kdc[2106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:41 dist-test-slave-2lzq krb5kdc[2106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
W20260929 10:59:41.420997 2201 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:41.415528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59628 (local address 127.26.120.254:43649)
0929 10:59:41.415632 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:41.415634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:41.415682 (+ 48us) server_negotiation.cc:407] Connection header received
0929 10:59:41.415764 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:41.415765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:41.415780 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:41.415857 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:41.416441 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.416929 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.417739 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.417913 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.417964 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.420939 (+ 2975us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59628: BlockingRecv error: recv got EOF from 127.0.0.1:59628 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Sep 29 10:59:41 dist-test-slave-2lzq krb5kdc[2106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679581, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM
I20260929 10:59:41.428385 2201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.422030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59640 (local address 127.26.120.254:43649)
0929 10:59:41.422129 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:41.422131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:41.422192 (+ 61us) server_negotiation.cc:407] Connection header received
0929 10:59:41.422250 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:41.422250 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:41.422261 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:41.422315 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:41.422804 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.423323 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.424010 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.424222 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.424299 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.426353 (+ 2054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:41.426354 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:41.426355 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:41.426361 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:41.427521 (+ 1160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:41.427816 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:41.427817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:41.427817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:41.427863 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:41.428028 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:41.428029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:41.428029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:41.428120 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:41.428191 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:41.428273 (+ 82us) server_negotiation.cc:299] Negotiation successful
0929 10:59:41.428324 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260929 10:59:41.429905 2151 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59640:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:59:41.431540 2151 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:59:41.437983 2656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.433750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:46459 (local address 127.0.0.1:33744)
0929 10:59:41.433917 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:41.433961 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:41.434027 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:41.434617 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:41.434618 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:41.434620 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:41.434674 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.434676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.435943 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.435944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.436989 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.436991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.437645 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.437646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.437727 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.437743 (+ 16us) client_negotiation.cc:769] Sending connection context
0929 10:59:41.437817 (+ 74us) client_negotiation.cc:240] Negotiation successful
0929 10:59:41.437868 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":48,"threads_started":1}
I20260929 10:59:41.437973 2660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.434097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33744 (local address 127.26.120.193:46459)
0929 10:59:41.434452 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:41.434457 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:41.434465 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:59:41.434498 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:41.434501 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:41.434515 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:41.434748 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:41.434804 (+ 56us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.435852 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.437066 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.437552 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.437667 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.437723 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:41.437831 (+ 108us) server_negotiation.cc:299] Negotiation successful
0929 10:59:41.437868 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":71,"threads_started":1}
I20260929 10:59:41.437992 2658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.433799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:45525 (local address 127.0.0.1:41072)
0929 10:59:41.433952 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:41.433962 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:41.434027 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:41.434431 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:41.434434 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:41.434436 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:41.434538 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.434540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.435503 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.435504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.436694 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.436696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.437734 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.437735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.437799 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.437803 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:59:41.437847 (+ 44us) client_negotiation.cc:240] Negotiation successful
0929 10:59:41.437888 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":1,"thread_start_us":98,"threads_started":1}
I20260929 10:59:41.438346 2659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.434044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41072 (local address 127.26.120.194:45525)
0929 10:59:41.434255 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:41.434263 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:41.434272 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:59:41.434313 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:41.434317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:41.434336 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:41.434425 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:41.434627 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.435406 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.436789 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.437567 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.437650 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.437702 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:41.438030 (+ 328us) server_negotiation.cc:299] Negotiation successful
0929 10:59:41.438231 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":102,"threads_started":1}
I20260929 10:59:41.439045 2428 tablet_service.cc:1467] Processing CreateTablet for tablet 626f29748de84bd49ec65ed5f0d89b2d (DEFAULT_TABLE table=test-table [id=e9a8f3f5a40d47b785eec6a96ae6a620]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:41.439224 2288 tablet_service.cc:1467] Processing CreateTablet for tablet 626f29748de84bd49ec65ed5f0d89b2d (DEFAULT_TABLE table=test-table [id=e9a8f3f5a40d47b785eec6a96ae6a620]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:41.439419 2428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 626f29748de84bd49ec65ed5f0d89b2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:41.439524 2288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 626f29748de84bd49ec65ed5f0d89b2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:41.439555 2657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.433799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:35405 (local address 127.0.0.1:42314)
0929 10:59:41.434155 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:41.434164 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:41.434206 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:41.434639 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:41.434639 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:41.434640 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:41.434674 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.434676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.437087 (+ 2411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.437088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.438277 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.438281 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.439306 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.439306 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.439370 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.439375 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 10:59:41.439424 (+ 49us) client_negotiation.cc:240] Negotiation successful
0929 10:59:41.439462 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":2,"thread_start_us":93,"threads_started":1}
I20260929 10:59:41.439865 2661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.434258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42314 (local address 127.26.120.195:35405)
0929 10:59:41.434410 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:41.434415 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:41.434426 (+ 11us) server_negotiation.cc:407] Connection header received
0929 10:59:41.434467 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:41.434470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:41.434485 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:41.434580 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:41.434751 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.435847 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.438374 (+ 2527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.439165 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.439282 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.439336 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:41.439728 (+ 392us) server_negotiation.cc:299] Negotiation successful
0929 10:59:41.439768 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":87,"threads_started":1}
I20260929 10:59:41.440387 2568 tablet_service.cc:1467] Processing CreateTablet for tablet 626f29748de84bd49ec65ed5f0d89b2d (DEFAULT_TABLE table=test-table [id=e9a8f3f5a40d47b785eec6a96ae6a620]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:41.440703 2568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 626f29748de84bd49ec65ed5f0d89b2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:41.443477 2662 tablet_bootstrap.cc:492] T 626f29748de84bd49ec65ed5f0d89b2d P 297c51b2364b4beb90b2d1f5282ac70a: Bootstrap starting.
I20260929 10:59:41.443691 2663 tablet_bootstrap.cc:492] T 626f29748de84bd49ec65ed5f0d89b2d P cff46e36bd9e400087ced7da2dbf9f0e: Bootstrap starting.
I20260929 10:59:41.443774 2664 tablet_bootstrap.cc:492] T 626f29748de84bd49ec65ed5f0d89b2d P c32b27909a9949f48f3ea22489973ace: Bootstrap starting.
I20260929 10:59:41.444263 2663 tablet_bootstrap.cc:654] T 626f29748de84bd49ec65ed5f0d89b2d P cff46e36bd9e400087ced7da2dbf9f0e: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:41.444386 2664 tablet_bootstrap.cc:654] T 626f29748de84bd49ec65ed5f0d89b2d P c32b27909a9949f48f3ea22489973ace: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:41.444604 2663 log.cc:824] T 626f29748de84bd49ec65ed5f0d89b2d P cff46e36bd9e400087ced7da2dbf9f0e: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:41.444708 2664 log.cc:824] T 626f29748de84bd49ec65ed5f0d89b2d P c32b27909a9949f48f3ea22489973ace: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:41.445358 2663 tablet_bootstrap.cc:492] T 626f29748de84bd49ec65ed5f0d89b2d P cff46e36bd9e400087ced7da2dbf9f0e: No bootstrap required, opened a new log
I20260929 10:59:41.445469 2662 tablet_bootstrap.cc:654] T 626f29748de84bd49ec65ed5f0d89b2d P 297c51b2364b4beb90b2d1f5282ac70a: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:41.445523 2664 tablet_bootstrap.cc:492] T 626f29748de84bd49ec65ed5f0d89b2d P c32b27909a9949f48f3ea22489973ace: No bootstrap required, opened a new log
I20260929 10:59:41.445601 2664 ts_tablet_manager.cc:1397] T 626f29748de84bd49ec65ed5f0d89b2d P c32b27909a9949f48f3ea22489973ace: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 10:59:41.445489 2663 ts_tablet_manager.cc:1397] T 626f29748de84bd49ec65ed5f0d89b2d P cff46e36bd9e400087ced7da2dbf9f0e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 10:59:41.445784 2662 log.cc:824] T 626f29748de84bd49ec65ed5f0d89b2d P 297c51b2364b4beb90b2d1f5282ac70a: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:41.446563 2662 tablet_bootstrap.cc:492] T 626f29748de84bd49ec65ed5f0d89b2d P 297c51b2364b4beb90b2d1f5282ac70a: No bootstrap required, opened a new log
I20260929 10:59:41.446625 2662 ts_tablet_manager.cc:1397] T 626f29748de84bd49ec65ed5f0d89b2d P 297c51b2364b4beb90b2d1f5282ac70a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260929 10:59:41.447393 2664 raft_consensus.cc:348] T 626f29748de84bd49ec65ed5f0d89b2d P c32b27909a9949f48f3ea22489973ace [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } }
I20260929 10:59:41.447559 2664 raft_consensus.cc:374] T 626f29748de84bd49ec65ed5f0d89b2d P c32b27909a9949f48f3ea22489973ace [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:41.447604 2664 raft_consensus.cc:729] T 626f29748de84bd49ec65ed5f0d89b2d P c32b27909a9949f48f3ea22489973ace [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c32b27909a9949f48f3ea22489973ace, State: Initialized, Role: FOLLOWER
I20260929 10:59:41.447734 2664 consensus_queue.cc:260] T 626f29748de84bd49ec65ed5f0d89b2d P c32b27909a9949f48f3ea22489973ace [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } }
I20260929 10:59:41.447707 2663 raft_consensus.cc:348] T 626f29748de84bd49ec65ed5f0d89b2d P cff46e36bd9e400087ced7da2dbf9f0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } }
I20260929 10:59:41.447870 2663 raft_consensus.cc:374] T 626f29748de84bd49ec65ed5f0d89b2d P cff46e36bd9e400087ced7da2dbf9f0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:41.447916 2663 raft_consensus.cc:729] T 626f29748de84bd49ec65ed5f0d89b2d P cff46e36bd9e400087ced7da2dbf9f0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cff46e36bd9e400087ced7da2dbf9f0e, State: Initialized, Role: FOLLOWER
I20260929 10:59:41.447988 2664 ts_tablet_manager.cc:1428] T 626f29748de84bd49ec65ed5f0d89b2d P c32b27909a9949f48f3ea22489973ace: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260929 10:59:41.448030 2663 consensus_queue.cc:260] T 626f29748de84bd49ec65ed5f0d89b2d P cff46e36bd9e400087ced7da2dbf9f0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } }
I20260929 10:59:41.448117 2634 heartbeater.cc:503] Master 127.26.120.254:43649 was elected leader, sending a full tablet report...
I20260929 10:59:41.448278 2663 ts_tablet_manager.cc:1428] T 626f29748de84bd49ec65ed5f0d89b2d P cff46e36bd9e400087ced7da2dbf9f0e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 10:59:41.448298 2494 heartbeater.cc:503] Master 127.26.120.254:43649 was elected leader, sending a full tablet report...
I20260929 10:59:41.448678 2662 raft_consensus.cc:348] T 626f29748de84bd49ec65ed5f0d89b2d P 297c51b2364b4beb90b2d1f5282ac70a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } }
I20260929 10:59:41.448805 2662 raft_consensus.cc:374] T 626f29748de84bd49ec65ed5f0d89b2d P 297c51b2364b4beb90b2d1f5282ac70a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:41.448827 2662 raft_consensus.cc:729] T 626f29748de84bd49ec65ed5f0d89b2d P 297c51b2364b4beb90b2d1f5282ac70a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297c51b2364b4beb90b2d1f5282ac70a, State: Initialized, Role: FOLLOWER
I20260929 10:59:41.448938 2662 consensus_queue.cc:260] T 626f29748de84bd49ec65ed5f0d89b2d P 297c51b2364b4beb90b2d1f5282ac70a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } }
I20260929 10:59:41.449204 2662 ts_tablet_manager.cc:1428] T 626f29748de84bd49ec65ed5f0d89b2d P 297c51b2364b4beb90b2d1f5282ac70a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 10:59:41.449309 2354 heartbeater.cc:503] Master 127.26.120.254:43649 was elected leader, sending a full tablet report...
I20260929 10:59:41.492353 2669 raft_consensus.cc:482] T 626f29748de84bd49ec65ed5f0d89b2d P cff46e36bd9e400087ced7da2dbf9f0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260929 10:59:41.492489 2495 tablet.cc:2369] T 626f29748de84bd49ec65ed5f0d89b2d P cff46e36bd9e400087ced7da2dbf9f0e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:59:41.492496 2669 raft_consensus.cc:504] T 626f29748de84bd49ec65ed5f0d89b2d P cff46e36bd9e400087ced7da2dbf9f0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } }
I20260929 10:59:41.492961 2669 leader_election.cc:288] T 626f29748de84bd49ec65ed5f0d89b2d P cff46e36bd9e400087ced7da2dbf9f0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297c51b2364b4beb90b2d1f5282ac70a (127.26.120.193:46459), c32b27909a9949f48f3ea22489973ace (127.26.120.195:35405)
I20260929 10:59:41.496809 2660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.493222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:45291 (local address 127.26.120.193:46459)
0929 10:59:41.493303 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:41.493305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:41.493341 (+ 36us) server_negotiation.cc:407] Connection header received
0929 10:59:41.493412 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:41.493413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:41.493428 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:41.493466 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:41.493637 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.494369 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.495897 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.496456 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.496529 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.496539 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:41.496698 (+ 159us) server_negotiation.cc:299] Negotiation successful
0929 10:59:41.496742 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 10:59:41.496824 2661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.493205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:40717 (local address 127.26.120.195:35405)
0929 10:59:41.493256 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:41.493258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:41.493266 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:59:41.493293 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:41.493294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:41.493305 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:41.493369 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:41.493574 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.494383 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.495874 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.496456 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.496538 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.496546 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:41.496718 (+ 172us) server_negotiation.cc:299] Negotiation successful
0929 10:59:41.496751 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260929 10:59:41.496832 2387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.493120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:35405 (local address 127.26.120.194:40717)
0929 10:59:41.493174 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:41.493185 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:41.493225 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:41.493390 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:41.493391 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:41.493393 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:41.493495 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.493497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.494458 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.494459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.495787 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.495790 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.496547 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.496548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.496620 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.496621 (+ 1us) client_negotiation.cc:769] Sending connection context
0929 10:59:41.496680 (+ 59us) client_negotiation.cc:240] Negotiation successful
0929 10:59:41.496730 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":44}
I20260929 10:59:41.496848 2671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.493137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:46459 (local address 127.26.120.194:45291)
0929 10:59:41.493292 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:41.493300 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:41.493349 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:41.493493 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:41.493494 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:41.493495 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:41.493559 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.493561 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.494458 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.494459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.495787 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.495790 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.496533 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.496534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.496615 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.496620 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 10:59:41.496680 (+ 60us) client_negotiation.cc:240] Negotiation successful
0929 10:59:41.496729 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":72,"thread_start_us":108,"threads_started":1}
I20260929 10:59:41.497216 2308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "626f29748de84bd49ec65ed5f0d89b2d" candidate_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297c51b2364b4beb90b2d1f5282ac70a" is_pre_election: true
I20260929 10:59:41.497215 2588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "626f29748de84bd49ec65ed5f0d89b2d" candidate_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c32b27909a9949f48f3ea22489973ace" is_pre_election: true
I20260929 10:59:41.497355 2308 raft_consensus.cc:2454] T 626f29748de84bd49ec65ed5f0d89b2d P 297c51b2364b4beb90b2d1f5282ac70a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cff46e36bd9e400087ced7da2dbf9f0e in term 0.
I20260929 10:59:41.497365 2588 raft_consensus.cc:2454] T 626f29748de84bd49ec65ed5f0d89b2d P c32b27909a9949f48f3ea22489973ace [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cff46e36bd9e400087ced7da2dbf9f0e in term 0.
I20260929 10:59:41.497561 2379 leader_election.cc:302] T 626f29748de84bd49ec65ed5f0d89b2d P cff46e36bd9e400087ced7da2dbf9f0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297c51b2364b4beb90b2d1f5282ac70a, cff46e36bd9e400087ced7da2dbf9f0e; no voters:
I20260929 10:59:41.497709 2669 raft_consensus.cc:2781] T 626f29748de84bd49ec65ed5f0d89b2d P cff46e36bd9e400087ced7da2dbf9f0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:59:41.497769 2669 raft_consensus.cc:482] T 626f29748de84bd49ec65ed5f0d89b2d P cff46e36bd9e400087ced7da2dbf9f0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:59:41.497789 2669 raft_consensus.cc:3037] T 626f29748de84bd49ec65ed5f0d89b2d P cff46e36bd9e400087ced7da2dbf9f0e [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:41.498859 2669 raft_consensus.cc:504] T 626f29748de84bd49ec65ed5f0d89b2d P cff46e36bd9e400087ced7da2dbf9f0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } }
I20260929 10:59:41.499001 2669 leader_election.cc:288] T 626f29748de84bd49ec65ed5f0d89b2d P cff46e36bd9e400087ced7da2dbf9f0e [CANDIDATE]: Term 1 election: Requested vote from peers 297c51b2364b4beb90b2d1f5282ac70a (127.26.120.193:46459), c32b27909a9949f48f3ea22489973ace (127.26.120.195:35405)
I20260929 10:59:41.499148 2308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "626f29748de84bd49ec65ed5f0d89b2d" candidate_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297c51b2364b4beb90b2d1f5282ac70a"
I20260929 10:59:41.499186 2588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "626f29748de84bd49ec65ed5f0d89b2d" candidate_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c32b27909a9949f48f3ea22489973ace"
I20260929 10:59:41.499257 2308 raft_consensus.cc:3037] T 626f29748de84bd49ec65ed5f0d89b2d P 297c51b2364b4beb90b2d1f5282ac70a [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:41.499251 2588 raft_consensus.cc:3037] T 626f29748de84bd49ec65ed5f0d89b2d P c32b27909a9949f48f3ea22489973ace [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:41.500031 2308 raft_consensus.cc:2454] T 626f29748de84bd49ec65ed5f0d89b2d P 297c51b2364b4beb90b2d1f5282ac70a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cff46e36bd9e400087ced7da2dbf9f0e in term 1.
I20260929 10:59:41.500038 2588 raft_consensus.cc:2454] T 626f29748de84bd49ec65ed5f0d89b2d P c32b27909a9949f48f3ea22489973ace [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cff46e36bd9e400087ced7da2dbf9f0e in term 1.
I20260929 10:59:41.500206 2379 leader_election.cc:302] T 626f29748de84bd49ec65ed5f0d89b2d P cff46e36bd9e400087ced7da2dbf9f0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297c51b2364b4beb90b2d1f5282ac70a, cff46e36bd9e400087ced7da2dbf9f0e; no voters:
I20260929 10:59:41.500342 2669 raft_consensus.cc:2781] T 626f29748de84bd49ec65ed5f0d89b2d P cff46e36bd9e400087ced7da2dbf9f0e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:59:41.500530 2669 raft_consensus.cc:686] T 626f29748de84bd49ec65ed5f0d89b2d P cff46e36bd9e400087ced7da2dbf9f0e [term 1 LEADER]: Becoming Leader. State: Replica: cff46e36bd9e400087ced7da2dbf9f0e, State: Running, Role: LEADER
I20260929 10:59:41.500636 2669 consensus_queue.cc:237] T 626f29748de84bd49ec65ed5f0d89b2d P cff46e36bd9e400087ced7da2dbf9f0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } }
I20260929 10:59:41.501461 2151 catalog_manager.cc:5495] T 626f29748de84bd49ec65ed5f0d89b2d P cff46e36bd9e400087ced7da2dbf9f0e reported cstate change: term changed from 0 to 1, leader changed from <none> to cff46e36bd9e400087ced7da2dbf9f0e (127.26.120.194). New cstate: current_term: 1 leader_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } health_report { overall_health: HEALTHY } } }
I20260929 10:59:41.510385 2659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.507915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41086 (local address 127.26.120.194:45525)
0929 10:59:41.507980 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:41.507982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:41.507991 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:59:41.508022 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:41.508023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:41.508051 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:41.508138 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:59:41.508290 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.508754 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.509512 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.509731 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.509813 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.509998 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:59:41.510173 (+ 175us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:59:41.510265 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:41.510288 (+ 23us) server_negotiation.cc:299] Negotiation successful
0929 10:59:41.510309 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 10:59:41.511898 2308 raft_consensus.cc:1261] T 626f29748de84bd49ec65ed5f0d89b2d P 297c51b2364b4beb90b2d1f5282ac70a [term 1 FOLLOWER]: Refusing update from remote peer cff46e36bd9e400087ced7da2dbf9f0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:59:41.511904 2588 raft_consensus.cc:1261] T 626f29748de84bd49ec65ed5f0d89b2d P c32b27909a9949f48f3ea22489973ace [term 1 FOLLOWER]: Refusing update from remote peer cff46e36bd9e400087ced7da2dbf9f0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:59:41.512225 2669 consensus_queue.cc:1035] T 626f29748de84bd49ec65ed5f0d89b2d P cff46e36bd9e400087ced7da2dbf9f0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:59:41.512363 2672 consensus_queue.cc:1035] T 626f29748de84bd49ec65ed5f0d89b2d P cff46e36bd9e400087ced7da2dbf9f0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:59:41.514783 2676 mvcc.cc:205] Tried to move back new op lower bound from 7334623565870530560 to 7334623565827309568. Current Snapshot: MvccSnapshot[applied={T|T < 7334623565870530560}]
I20260929 10:59:41.515441 2677 mvcc.cc:205] Tried to move back new op lower bound from 7334623565870530560 to 7334623565827309568. Current Snapshot: MvccSnapshot[applied={T|T < 7334623565870530560}]
I20260929 10:59:41.516498 2150 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:59:41.516590 2150 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:59:41.517335 2150 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5b33e54e771e4830afa648431a0880a2: Sending DeleteTablet for 3 replicas of tablet 626f29748de84bd49ec65ed5f0d89b2d
I20260929 10:59:41.517673 2428 tablet_service.cc:1514] Processing DeleteTablet for tablet 626f29748de84bd49ec65ed5f0d89b2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:41 UTC) from {username='oryx', principal='oryx/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:41072
I20260929 10:59:41.517673 2568 tablet_service.cc:1514] Processing DeleteTablet for tablet 626f29748de84bd49ec65ed5f0d89b2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:41 UTC) from {username='oryx', principal='oryx/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:42314
I20260929 10:59:41.517894 2685 tablet_replica.cc:330] T 626f29748de84bd49ec65ed5f0d89b2d P c32b27909a9949f48f3ea22489973ace: stopping tablet replica
I20260929 10:59:41.518010 2685 raft_consensus.cc:2229] T 626f29748de84bd49ec65ed5f0d89b2d P c32b27909a9949f48f3ea22489973ace [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:59:41.518013 2288 tablet_service.cc:1514] Processing DeleteTablet for tablet 626f29748de84bd49ec65ed5f0d89b2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:41 UTC) from {username='oryx', principal='oryx/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:33744
I20260929 10:59:41.518407 2685 raft_consensus.cc:2258] T 626f29748de84bd49ec65ed5f0d89b2d P c32b27909a9949f48f3ea22489973ace [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:41.518538 2687 tablet_replica.cc:330] T 626f29748de84bd49ec65ed5f0d89b2d P 297c51b2364b4beb90b2d1f5282ac70a: stopping tablet replica
I20260929 10:59:41.518473 2150 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59640:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 10:59:41.518592 2686 tablet_replica.cc:330] T 626f29748de84bd49ec65ed5f0d89b2d P cff46e36bd9e400087ced7da2dbf9f0e: stopping tablet replica
W20260929 10:59:41.518643 2150 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:59:41.518723 2686 raft_consensus.cc:2229] T 626f29748de84bd49ec65ed5f0d89b2d P cff46e36bd9e400087ced7da2dbf9f0e [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:59:41.518808 2686 raft_consensus.cc:2258] T 626f29748de84bd49ec65ed5f0d89b2d P cff46e36bd9e400087ced7da2dbf9f0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:41.518903 2687 raft_consensus.cc:2229] T 626f29748de84bd49ec65ed5f0d89b2d P 297c51b2364b4beb90b2d1f5282ac70a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:59:41.519057 2687 raft_consensus.cc:2258] T 626f29748de84bd49ec65ed5f0d89b2d P 297c51b2364b4beb90b2d1f5282ac70a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:41.519285 2685 ts_tablet_manager.cc:1905] T 626f29748de84bd49ec65ed5f0d89b2d P c32b27909a9949f48f3ea22489973ace: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:41.519677 2687 ts_tablet_manager.cc:1905] T 626f29748de84bd49ec65ed5f0d89b2d P 297c51b2364b4beb90b2d1f5282ac70a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:41.520071 2428 tablet_service.cc:1467] Processing CreateTablet for tablet c8e4b7b22cb24e72ab12f65860f83056 (DEFAULT_TABLE table=test-table [id=aea4ba9562144f6b81e8991f53217f95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:41.520205 2428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8e4b7b22cb24e72ab12f65860f83056. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:41.520949 2685 ts_tablet_manager.cc:1918] T 626f29748de84bd49ec65ed5f0d89b2d P c32b27909a9949f48f3ea22489973ace: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:59:41.521031 2685 log.cc:1196] T 626f29748de84bd49ec65ed5f0d89b2d P c32b27909a9949f48f3ea22489973ace: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-2/wal/wals/626f29748de84bd49ec65ed5f0d89b2d
I20260929 10:59:41.521301 2685 ts_tablet_manager.cc:1939] T 626f29748de84bd49ec65ed5f0d89b2d P c32b27909a9949f48f3ea22489973ace: Deleting consensus metadata
I20260929 10:59:41.521621 2687 ts_tablet_manager.cc:1918] T 626f29748de84bd49ec65ed5f0d89b2d P 297c51b2364b4beb90b2d1f5282ac70a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:59:41.521646 2663 tablet_bootstrap.cc:492] T c8e4b7b22cb24e72ab12f65860f83056 P cff46e36bd9e400087ced7da2dbf9f0e: Bootstrap starting.
I20260929 10:59:41.521756 2687 log.cc:1196] T 626f29748de84bd49ec65ed5f0d89b2d P 297c51b2364b4beb90b2d1f5282ac70a: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-0/wal/wals/626f29748de84bd49ec65ed5f0d89b2d
I20260929 10:59:41.522045 2663 tablet_bootstrap.cc:654] T c8e4b7b22cb24e72ab12f65860f83056 P cff46e36bd9e400087ced7da2dbf9f0e: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:41.522038 2288 tablet_service.cc:1467] Processing CreateTablet for tablet c8e4b7b22cb24e72ab12f65860f83056 (DEFAULT_TABLE table=test-table [id=aea4ba9562144f6b81e8991f53217f95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:41.522199 2288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8e4b7b22cb24e72ab12f65860f83056. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:41.522327 2687 ts_tablet_manager.cc:1939] T 626f29748de84bd49ec65ed5f0d89b2d P 297c51b2364b4beb90b2d1f5282ac70a: Deleting consensus metadata
I20260929 10:59:41.522773 2663 tablet_bootstrap.cc:492] T c8e4b7b22cb24e72ab12f65860f83056 P cff46e36bd9e400087ced7da2dbf9f0e: No bootstrap required, opened a new log
I20260929 10:59:41.522848 2663 ts_tablet_manager.cc:1397] T c8e4b7b22cb24e72ab12f65860f83056 P cff46e36bd9e400087ced7da2dbf9f0e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:59:41.522990 2663 raft_consensus.cc:348] T c8e4b7b22cb24e72ab12f65860f83056 P cff46e36bd9e400087ced7da2dbf9f0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } }
I20260929 10:59:41.523084 2663 raft_consensus.cc:374] T c8e4b7b22cb24e72ab12f65860f83056 P cff46e36bd9e400087ced7da2dbf9f0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:41.523111 2663 raft_consensus.cc:729] T c8e4b7b22cb24e72ab12f65860f83056 P cff46e36bd9e400087ced7da2dbf9f0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cff46e36bd9e400087ced7da2dbf9f0e, State: Initialized, Role: FOLLOWER
I20260929 10:59:41.523154 2663 consensus_queue.cc:260] T c8e4b7b22cb24e72ab12f65860f83056 P cff46e36bd9e400087ced7da2dbf9f0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } }
I20260929 10:59:41.523245 2663 ts_tablet_manager.cc:1428] T c8e4b7b22cb24e72ab12f65860f83056 P cff46e36bd9e400087ced7da2dbf9f0e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260929 10:59:41.523371 2662 tablet_bootstrap.cc:492] T c8e4b7b22cb24e72ab12f65860f83056 P 297c51b2364b4beb90b2d1f5282ac70a: Bootstrap starting.
I20260929 10:59:41.523801 2662 tablet_bootstrap.cc:654] T c8e4b7b22cb24e72ab12f65860f83056 P 297c51b2364b4beb90b2d1f5282ac70a: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:41.524175 2135 catalog_manager.cc:4841] TS 297c51b2364b4beb90b2d1f5282ac70a (127.26.120.193:46459): tablet 626f29748de84bd49ec65ed5f0d89b2d (table test-table [id=e9a8f3f5a40d47b785eec6a96ae6a620]) successfully deleted
I20260929 10:59:41.524226 2137 catalog_manager.cc:4841] TS c32b27909a9949f48f3ea22489973ace (127.26.120.195:35405): tablet 626f29748de84bd49ec65ed5f0d89b2d (table test-table [id=e9a8f3f5a40d47b785eec6a96ae6a620]) successfully deleted
I20260929 10:59:41.524366 2568 tablet_service.cc:1467] Processing CreateTablet for tablet c8e4b7b22cb24e72ab12f65860f83056 (DEFAULT_TABLE table=test-table [id=aea4ba9562144f6b81e8991f53217f95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:41.524473 2568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8e4b7b22cb24e72ab12f65860f83056. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:41.525663 2664 tablet_bootstrap.cc:492] T c8e4b7b22cb24e72ab12f65860f83056 P c32b27909a9949f48f3ea22489973ace: Bootstrap starting.
I20260929 10:59:41.525900 2662 tablet_bootstrap.cc:492] T c8e4b7b22cb24e72ab12f65860f83056 P 297c51b2364b4beb90b2d1f5282ac70a: No bootstrap required, opened a new log
I20260929 10:59:41.525956 2662 ts_tablet_manager.cc:1397] T c8e4b7b22cb24e72ab12f65860f83056 P 297c51b2364b4beb90b2d1f5282ac70a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260929 10:59:41.526043 2664 tablet_bootstrap.cc:654] T c8e4b7b22cb24e72ab12f65860f83056 P c32b27909a9949f48f3ea22489973ace: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:41.526132 2662 raft_consensus.cc:348] T c8e4b7b22cb24e72ab12f65860f83056 P 297c51b2364b4beb90b2d1f5282ac70a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } }
I20260929 10:59:41.526206 2662 raft_consensus.cc:374] T c8e4b7b22cb24e72ab12f65860f83056 P 297c51b2364b4beb90b2d1f5282ac70a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:41.526227 2662 raft_consensus.cc:729] T c8e4b7b22cb24e72ab12f65860f83056 P 297c51b2364b4beb90b2d1f5282ac70a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297c51b2364b4beb90b2d1f5282ac70a, State: Initialized, Role: FOLLOWER
I20260929 10:59:41.526266 2662 consensus_queue.cc:260] T c8e4b7b22cb24e72ab12f65860f83056 P 297c51b2364b4beb90b2d1f5282ac70a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } }
I20260929 10:59:41.526355 2662 ts_tablet_manager.cc:1428] T c8e4b7b22cb24e72ab12f65860f83056 P 297c51b2364b4beb90b2d1f5282ac70a: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260929 10:59:41.526624 2686 ts_tablet_manager.cc:1905] T 626f29748de84bd49ec65ed5f0d89b2d P cff46e36bd9e400087ced7da2dbf9f0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:41.526763 2664 tablet_bootstrap.cc:492] T c8e4b7b22cb24e72ab12f65860f83056 P c32b27909a9949f48f3ea22489973ace: No bootstrap required, opened a new log
I20260929 10:59:41.526805 2664 ts_tablet_manager.cc:1397] T c8e4b7b22cb24e72ab12f65860f83056 P c32b27909a9949f48f3ea22489973ace: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 10:59:41.526933 2664 raft_consensus.cc:348] T c8e4b7b22cb24e72ab12f65860f83056 P c32b27909a9949f48f3ea22489973ace [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } }
I20260929 10:59:41.526989 2664 raft_consensus.cc:374] T c8e4b7b22cb24e72ab12f65860f83056 P c32b27909a9949f48f3ea22489973ace [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:41.527009 2664 raft_consensus.cc:729] T c8e4b7b22cb24e72ab12f65860f83056 P c32b27909a9949f48f3ea22489973ace [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c32b27909a9949f48f3ea22489973ace, State: Initialized, Role: FOLLOWER
I20260929 10:59:41.527060 2664 consensus_queue.cc:260] T c8e4b7b22cb24e72ab12f65860f83056 P c32b27909a9949f48f3ea22489973ace [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } }
I20260929 10:59:41.527135 2664 ts_tablet_manager.cc:1428] T c8e4b7b22cb24e72ab12f65860f83056 P c32b27909a9949f48f3ea22489973ace: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:41.528146 2686 ts_tablet_manager.cc:1918] T 626f29748de84bd49ec65ed5f0d89b2d P cff46e36bd9e400087ced7da2dbf9f0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:59:41.528230 2686 log.cc:1196] T 626f29748de84bd49ec65ed5f0d89b2d P cff46e36bd9e400087ced7da2dbf9f0e: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-1/wal/wals/626f29748de84bd49ec65ed5f0d89b2d
I20260929 10:59:41.528517 2686 ts_tablet_manager.cc:1939] T 626f29748de84bd49ec65ed5f0d89b2d P cff46e36bd9e400087ced7da2dbf9f0e: Deleting consensus metadata
I20260929 10:59:41.528874 2138 catalog_manager.cc:4841] TS cff46e36bd9e400087ced7da2dbf9f0e (127.26.120.194:45525): tablet 626f29748de84bd49ec65ed5f0d89b2d (table test-table [id=e9a8f3f5a40d47b785eec6a96ae6a620]) successfully deleted
W20260929 10:59:41.586524 2355 tablet.cc:2369] T c8e4b7b22cb24e72ab12f65860f83056 P 297c51b2364b4beb90b2d1f5282ac70a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 10:59:41.643723 2635 tablet.cc:2369] T c8e4b7b22cb24e72ab12f65860f83056 P c32b27909a9949f48f3ea22489973ace: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 10:59:41.672837 2681 raft_consensus.cc:482] T c8e4b7b22cb24e72ab12f65860f83056 P cff46e36bd9e400087ced7da2dbf9f0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:59:41.672974 2681 raft_consensus.cc:504] T c8e4b7b22cb24e72ab12f65860f83056 P cff46e36bd9e400087ced7da2dbf9f0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } }
I20260929 10:59:41.673128 2681 leader_election.cc:288] T c8e4b7b22cb24e72ab12f65860f83056 P cff46e36bd9e400087ced7da2dbf9f0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297c51b2364b4beb90b2d1f5282ac70a (127.26.120.193:46459), c32b27909a9949f48f3ea22489973ace (127.26.120.195:35405)
I20260929 10:59:41.673311 2588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8e4b7b22cb24e72ab12f65860f83056" candidate_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c32b27909a9949f48f3ea22489973ace" is_pre_election: true
I20260929 10:59:41.673290 2308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8e4b7b22cb24e72ab12f65860f83056" candidate_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297c51b2364b4beb90b2d1f5282ac70a" is_pre_election: true
I20260929 10:59:41.673441 2588 raft_consensus.cc:2454] T c8e4b7b22cb24e72ab12f65860f83056 P c32b27909a9949f48f3ea22489973ace [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cff46e36bd9e400087ced7da2dbf9f0e in term 0.
I20260929 10:59:41.673439 2308 raft_consensus.cc:2454] T c8e4b7b22cb24e72ab12f65860f83056 P 297c51b2364b4beb90b2d1f5282ac70a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cff46e36bd9e400087ced7da2dbf9f0e in term 0.
I20260929 10:59:41.673609 2381 leader_election.cc:302] T c8e4b7b22cb24e72ab12f65860f83056 P cff46e36bd9e400087ced7da2dbf9f0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c32b27909a9949f48f3ea22489973ace, cff46e36bd9e400087ced7da2dbf9f0e; no voters:
I20260929 10:59:41.673715 2681 raft_consensus.cc:2781] T c8e4b7b22cb24e72ab12f65860f83056 P cff46e36bd9e400087ced7da2dbf9f0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:59:41.673767 2681 raft_consensus.cc:482] T c8e4b7b22cb24e72ab12f65860f83056 P cff46e36bd9e400087ced7da2dbf9f0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:59:41.673791 2681 raft_consensus.cc:3037] T c8e4b7b22cb24e72ab12f65860f83056 P cff46e36bd9e400087ced7da2dbf9f0e [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:41.674645 2681 raft_consensus.cc:504] T c8e4b7b22cb24e72ab12f65860f83056 P cff46e36bd9e400087ced7da2dbf9f0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } }
I20260929 10:59:41.674798 2681 leader_election.cc:288] T c8e4b7b22cb24e72ab12f65860f83056 P cff46e36bd9e400087ced7da2dbf9f0e [CANDIDATE]: Term 1 election: Requested vote from peers 297c51b2364b4beb90b2d1f5282ac70a (127.26.120.193:46459), c32b27909a9949f48f3ea22489973ace (127.26.120.195:35405)
I20260929 10:59:41.674944 2308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8e4b7b22cb24e72ab12f65860f83056" candidate_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297c51b2364b4beb90b2d1f5282ac70a"
I20260929 10:59:41.675001 2588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8e4b7b22cb24e72ab12f65860f83056" candidate_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c32b27909a9949f48f3ea22489973ace"
I20260929 10:59:41.675050 2308 raft_consensus.cc:3037] T c8e4b7b22cb24e72ab12f65860f83056 P 297c51b2364b4beb90b2d1f5282ac70a [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:41.675066 2588 raft_consensus.cc:3037] T c8e4b7b22cb24e72ab12f65860f83056 P c32b27909a9949f48f3ea22489973ace [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:41.675796 2588 raft_consensus.cc:2454] T c8e4b7b22cb24e72ab12f65860f83056 P c32b27909a9949f48f3ea22489973ace [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cff46e36bd9e400087ced7da2dbf9f0e in term 1.
I20260929 10:59:41.675796 2308 raft_consensus.cc:2454] T c8e4b7b22cb24e72ab12f65860f83056 P 297c51b2364b4beb90b2d1f5282ac70a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cff46e36bd9e400087ced7da2dbf9f0e in term 1.
I20260929 10:59:41.676004 2379 leader_election.cc:302] T c8e4b7b22cb24e72ab12f65860f83056 P cff46e36bd9e400087ced7da2dbf9f0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297c51b2364b4beb90b2d1f5282ac70a, cff46e36bd9e400087ced7da2dbf9f0e; no voters:
I20260929 10:59:41.676121 2681 raft_consensus.cc:2781] T c8e4b7b22cb24e72ab12f65860f83056 P cff46e36bd9e400087ced7da2dbf9f0e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:59:41.676208 2681 raft_consensus.cc:686] T c8e4b7b22cb24e72ab12f65860f83056 P cff46e36bd9e400087ced7da2dbf9f0e [term 1 LEADER]: Becoming Leader. State: Replica: cff46e36bd9e400087ced7da2dbf9f0e, State: Running, Role: LEADER
I20260929 10:59:41.676262 2681 consensus_queue.cc:237] T c8e4b7b22cb24e72ab12f65860f83056 P cff46e36bd9e400087ced7da2dbf9f0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } }
I20260929 10:59:41.676818 2150 catalog_manager.cc:5495] T c8e4b7b22cb24e72ab12f65860f83056 P cff46e36bd9e400087ced7da2dbf9f0e reported cstate change: term changed from 0 to 1, leader changed from <none> to cff46e36bd9e400087ced7da2dbf9f0e (127.26.120.194). New cstate: current_term: 1 leader_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } health_report { overall_health: UNKNOWN } } }
Sep 29 10:59:41 dist-test-slave-2lzq krb5kdc[2106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679580, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.254@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM
I20260929 10:59:41.733626 2201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.726817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59652 (local address 127.26.120.254:43649)
0929 10:59:41.726900 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:41.726903 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:41.727026 (+ 123us) server_negotiation.cc:407] Connection header received
0929 10:59:41.727132 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:41.727132 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:41.727142 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:41.727214 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:41.727701 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.728193 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.728930 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.729112 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.729197 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.731271 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:41.731272 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:41.731272 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:41.731279 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:41.732424 (+ 1145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:41.732815 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:41.732816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:41.732817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:41.732874 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:41.733074 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:41.733076 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:41.733076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:41.733184 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:41.733262 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:41.733415 (+ 153us) server_negotiation.cc:299] Negotiation successful
0929 10:59:41.733499 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 10:59:41.733692 2697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.726747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43649 (local address 127.0.0.1:59652)
0929 10:59:41.726949 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:41.726959 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:41.727048 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:41.727218 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:41.727219 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:41.727558 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:41.727613 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.727615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.728283 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.728284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.728805 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.728807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.729199 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.729200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.729254 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.729746 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:41.729750 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:41.731163 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:41.732529 (+ 1366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:41.732537 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:41.732540 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:41.732726 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:41.732942 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:41.732943 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:41.732943 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:41.733010 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:41.733252 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:41.733259 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:41.733303 (+ 44us) client_negotiation.cc:769] Sending connection context
0929 10:59:41.733390 (+ 87us) client_negotiation.cc:240] Negotiation successful
0929 10:59:41.733569 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":82,"thread_start_us":99,"threads_started":1}
I20260929 10:59:41.735533 2150 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:59652:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 10:59:41.735746 2150 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:59:41.737105 2288 tablet_service.cc:1467] Processing CreateTablet for tablet 4e5b51c3b76a4614a81c2226b30362bc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4b2af5e5c6c348cc921f252c4686d1d5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 10:59:41.737133 2568 tablet_service.cc:1467] Processing CreateTablet for tablet 4e5b51c3b76a4614a81c2226b30362bc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4b2af5e5c6c348cc921f252c4686d1d5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 10:59:41.737280 2568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e5b51c3b76a4614a81c2226b30362bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:41.737279 2288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e5b51c3b76a4614a81c2226b30362bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:41.738579 2662 tablet_bootstrap.cc:492] T 4e5b51c3b76a4614a81c2226b30362bc P 297c51b2364b4beb90b2d1f5282ac70a: Bootstrap starting.
I20260929 10:59:41.738615 2664 tablet_bootstrap.cc:492] T 4e5b51c3b76a4614a81c2226b30362bc P c32b27909a9949f48f3ea22489973ace: Bootstrap starting.
I20260929 10:59:41.738824 2428 tablet_service.cc:1467] Processing CreateTablet for tablet 4e5b51c3b76a4614a81c2226b30362bc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4b2af5e5c6c348cc921f252c4686d1d5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 10:59:41.738996 2428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e5b51c3b76a4614a81c2226b30362bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:41.739145 2662 tablet_bootstrap.cc:654] T 4e5b51c3b76a4614a81c2226b30362bc P 297c51b2364b4beb90b2d1f5282ac70a: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:41.739164 2664 tablet_bootstrap.cc:654] T 4e5b51c3b76a4614a81c2226b30362bc P c32b27909a9949f48f3ea22489973ace: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:41.739820 2662 tablet_bootstrap.cc:492] T 4e5b51c3b76a4614a81c2226b30362bc P 297c51b2364b4beb90b2d1f5282ac70a: No bootstrap required, opened a new log
I20260929 10:59:41.739893 2662 ts_tablet_manager.cc:1397] T 4e5b51c3b76a4614a81c2226b30362bc P 297c51b2364b4beb90b2d1f5282ac70a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 10:59:41.740015 2664 tablet_bootstrap.cc:492] T 4e5b51c3b76a4614a81c2226b30362bc P c32b27909a9949f48f3ea22489973ace: No bootstrap required, opened a new log
I20260929 10:59:41.740061 2664 ts_tablet_manager.cc:1397] T 4e5b51c3b76a4614a81c2226b30362bc P c32b27909a9949f48f3ea22489973ace: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 10:59:41.740075 2662 raft_consensus.cc:348] T 4e5b51c3b76a4614a81c2226b30362bc P 297c51b2364b4beb90b2d1f5282ac70a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } }
I20260929 10:59:41.740162 2662 raft_consensus.cc:374] T 4e5b51c3b76a4614a81c2226b30362bc P 297c51b2364b4beb90b2d1f5282ac70a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:41.740191 2662 raft_consensus.cc:729] T 4e5b51c3b76a4614a81c2226b30362bc P 297c51b2364b4beb90b2d1f5282ac70a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297c51b2364b4beb90b2d1f5282ac70a, State: Initialized, Role: FOLLOWER
I20260929 10:59:41.740213 2664 raft_consensus.cc:348] T 4e5b51c3b76a4614a81c2226b30362bc P c32b27909a9949f48f3ea22489973ace [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } }
I20260929 10:59:41.740233 2662 consensus_queue.cc:260] T 4e5b51c3b76a4614a81c2226b30362bc P 297c51b2364b4beb90b2d1f5282ac70a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } }
I20260929 10:59:41.740278 2664 raft_consensus.cc:374] T 4e5b51c3b76a4614a81c2226b30362bc P c32b27909a9949f48f3ea22489973ace [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:41.740304 2664 raft_consensus.cc:729] T 4e5b51c3b76a4614a81c2226b30362bc P c32b27909a9949f48f3ea22489973ace [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c32b27909a9949f48f3ea22489973ace, State: Initialized, Role: FOLLOWER
I20260929 10:59:41.740324 2662 ts_tablet_manager.cc:1428] T 4e5b51c3b76a4614a81c2226b30362bc P 297c51b2364b4beb90b2d1f5282ac70a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:41.740341 2664 consensus_queue.cc:260] T 4e5b51c3b76a4614a81c2226b30362bc P c32b27909a9949f48f3ea22489973ace [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } }
I20260929 10:59:41.740430 2664 ts_tablet_manager.cc:1428] T 4e5b51c3b76a4614a81c2226b30362bc P c32b27909a9949f48f3ea22489973ace: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:41.740496 2668 tablet_replica.cc:439] T 4e5b51c3b76a4614a81c2226b30362bc P c32b27909a9949f48f3ea22489973ace: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } } }
I20260929 10:59:41.740594 2668 tablet_replica.cc:442] T 4e5b51c3b76a4614a81c2226b30362bc P c32b27909a9949f48f3ea22489973ace: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:59:41.740619 2670 tablet_replica.cc:439] T 4e5b51c3b76a4614a81c2226b30362bc P 297c51b2364b4beb90b2d1f5282ac70a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } } }
I20260929 10:59:41.740734 2670 tablet_replica.cc:442] T 4e5b51c3b76a4614a81c2226b30362bc P 297c51b2364b4beb90b2d1f5282ac70a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:59:41.741052 2663 tablet_bootstrap.cc:492] T 4e5b51c3b76a4614a81c2226b30362bc P cff46e36bd9e400087ced7da2dbf9f0e: Bootstrap starting.
I20260929 10:59:41.741417 2663 tablet_bootstrap.cc:654] T 4e5b51c3b76a4614a81c2226b30362bc P cff46e36bd9e400087ced7da2dbf9f0e: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:41.741952 2663 tablet_bootstrap.cc:492] T 4e5b51c3b76a4614a81c2226b30362bc P cff46e36bd9e400087ced7da2dbf9f0e: No bootstrap required, opened a new log
I20260929 10:59:41.741992 2663 ts_tablet_manager.cc:1397] T 4e5b51c3b76a4614a81c2226b30362bc P cff46e36bd9e400087ced7da2dbf9f0e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:59:41.742136 2663 raft_consensus.cc:348] T 4e5b51c3b76a4614a81c2226b30362bc P cff46e36bd9e400087ced7da2dbf9f0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } }
I20260929 10:59:41.742201 2663 raft_consensus.cc:374] T 4e5b51c3b76a4614a81c2226b30362bc P cff46e36bd9e400087ced7da2dbf9f0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:41.742216 2663 raft_consensus.cc:729] T 4e5b51c3b76a4614a81c2226b30362bc P cff46e36bd9e400087ced7da2dbf9f0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cff46e36bd9e400087ced7da2dbf9f0e, State: Initialized, Role: FOLLOWER
I20260929 10:59:41.742269 2663 consensus_queue.cc:260] T 4e5b51c3b76a4614a81c2226b30362bc P cff46e36bd9e400087ced7da2dbf9f0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } }
I20260929 10:59:41.742367 2663 ts_tablet_manager.cc:1428] T 4e5b51c3b76a4614a81c2226b30362bc P cff46e36bd9e400087ced7da2dbf9f0e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:41.742379 2681 tablet_replica.cc:439] T 4e5b51c3b76a4614a81c2226b30362bc P cff46e36bd9e400087ced7da2dbf9f0e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } } }
I20260929 10:59:41.742450 2681 tablet_replica.cc:442] T 4e5b51c3b76a4614a81c2226b30362bc P cff46e36bd9e400087ced7da2dbf9f0e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:59:41.751695 2668 raft_consensus.cc:482] T 4e5b51c3b76a4614a81c2226b30362bc P c32b27909a9949f48f3ea22489973ace [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:59:41.751765 2668 raft_consensus.cc:504] T 4e5b51c3b76a4614a81c2226b30362bc P c32b27909a9949f48f3ea22489973ace [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } }
I20260929 10:59:41.752118 2668 leader_election.cc:288] T 4e5b51c3b76a4614a81c2226b30362bc P c32b27909a9949f48f3ea22489973ace [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297c51b2364b4beb90b2d1f5282ac70a (127.26.120.193:46459), cff46e36bd9e400087ced7da2dbf9f0e (127.26.120.194:45525)
I20260929 10:59:41.754973 2527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.752246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:45525 (local address 127.26.120.195:57213)
0929 10:59:41.752319 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:41.752332 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:41.752378 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:41.752521 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:41.752522 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:41.752524 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:41.752602 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.752604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.753272 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.753273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.754137 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.754140 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.754798 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.754798 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.754853 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.754856 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:59:41.754896 (+ 40us) client_negotiation.cc:240] Negotiation successful
0929 10:59:41.754924 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260929 10:59:41.755023 2659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.752322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:57213 (local address 127.26.120.194:45525)
0929 10:59:41.752384 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:41.752386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:41.752399 (+ 13us) server_negotiation.cc:407] Connection header received
0929 10:59:41.752425 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:41.752426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:41.752440 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:41.752521 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:41.752675 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.753193 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.754218 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.754725 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.754796 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.754806 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:41.754921 (+ 115us) server_negotiation.cc:299] Negotiation successful
0929 10:59:41.754959 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 10:59:41.755268 2448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e5b51c3b76a4614a81c2226b30362bc" candidate_uuid: "c32b27909a9949f48f3ea22489973ace" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" is_pre_election: true
I20260929 10:59:41.755394 2448 raft_consensus.cc:2454] T 4e5b51c3b76a4614a81c2226b30362bc P cff46e36bd9e400087ced7da2dbf9f0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c32b27909a9949f48f3ea22489973ace in term 0.
I20260929 10:59:41.755604 2522 leader_election.cc:302] T 4e5b51c3b76a4614a81c2226b30362bc P c32b27909a9949f48f3ea22489973ace [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c32b27909a9949f48f3ea22489973ace, cff46e36bd9e400087ced7da2dbf9f0e; no voters:
I20260929 10:59:41.755661 2660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.752431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:60455 (local address 127.26.120.193:46459)
0929 10:59:41.752497 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:41.752499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:41.752508 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:59:41.752531 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:41.752532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:41.752553 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:41.752613 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:41.752792 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.753289 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.754722 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.755313 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.755389 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.755397 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:41.755554 (+ 157us) server_negotiation.cc:299] Negotiation successful
0929 10:59:41.755595 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 10:59:41.755684 2701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.752259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:46459 (local address 127.26.120.195:60455)
0929 10:59:41.752425 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:41.752435 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:41.752482 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:41.752624 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:41.752625 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:41.752626 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:41.752720 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.752722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.753379 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.753380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.754638 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.754640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.755411 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.755412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.755484 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.755488 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 10:59:41.755544 (+ 56us) client_negotiation.cc:240] Negotiation successful
0929 10:59:41.755578 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":103,"threads_started":1}
I20260929 10:59:41.755762 2668 raft_consensus.cc:2781] T 4e5b51c3b76a4614a81c2226b30362bc P c32b27909a9949f48f3ea22489973ace [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:59:41.755801 2668 raft_consensus.cc:482] T 4e5b51c3b76a4614a81c2226b30362bc P c32b27909a9949f48f3ea22489973ace [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:59:41.755827 2668 raft_consensus.cc:3037] T 4e5b51c3b76a4614a81c2226b30362bc P c32b27909a9949f48f3ea22489973ace [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:41.756214 2308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e5b51c3b76a4614a81c2226b30362bc" candidate_uuid: "c32b27909a9949f48f3ea22489973ace" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297c51b2364b4beb90b2d1f5282ac70a" is_pre_election: true
I20260929 10:59:41.756305 2308 raft_consensus.cc:2454] T 4e5b51c3b76a4614a81c2226b30362bc P 297c51b2364b4beb90b2d1f5282ac70a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c32b27909a9949f48f3ea22489973ace in term 0.
I20260929 10:59:41.756686 2668 raft_consensus.cc:504] T 4e5b51c3b76a4614a81c2226b30362bc P c32b27909a9949f48f3ea22489973ace [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } }
I20260929 10:59:41.756827 2668 leader_election.cc:288] T 4e5b51c3b76a4614a81c2226b30362bc P c32b27909a9949f48f3ea22489973ace [CANDIDATE]: Term 1 election: Requested vote from peers 297c51b2364b4beb90b2d1f5282ac70a (127.26.120.193:46459), cff46e36bd9e400087ced7da2dbf9f0e (127.26.120.194:45525)
I20260929 10:59:41.756985 2308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e5b51c3b76a4614a81c2226b30362bc" candidate_uuid: "c32b27909a9949f48f3ea22489973ace" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297c51b2364b4beb90b2d1f5282ac70a"
I20260929 10:59:41.757066 2308 raft_consensus.cc:3037] T 4e5b51c3b76a4614a81c2226b30362bc P 297c51b2364b4beb90b2d1f5282ac70a [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:41.757608 2308 raft_consensus.cc:2454] T 4e5b51c3b76a4614a81c2226b30362bc P 297c51b2364b4beb90b2d1f5282ac70a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c32b27909a9949f48f3ea22489973ace in term 1.
I20260929 10:59:41.757733 2448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e5b51c3b76a4614a81c2226b30362bc" candidate_uuid: "c32b27909a9949f48f3ea22489973ace" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cff46e36bd9e400087ced7da2dbf9f0e"
I20260929 10:59:41.757807 2448 raft_consensus.cc:3037] T 4e5b51c3b76a4614a81c2226b30362bc P cff46e36bd9e400087ced7da2dbf9f0e [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:41.757817 2519 leader_election.cc:302] T 4e5b51c3b76a4614a81c2226b30362bc P c32b27909a9949f48f3ea22489973ace [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297c51b2364b4beb90b2d1f5282ac70a, c32b27909a9949f48f3ea22489973ace; no voters:
I20260929 10:59:41.757902 2668 raft_consensus.cc:2781] T 4e5b51c3b76a4614a81c2226b30362bc P c32b27909a9949f48f3ea22489973ace [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:59:41.758047 2668 raft_consensus.cc:686] T 4e5b51c3b76a4614a81c2226b30362bc P c32b27909a9949f48f3ea22489973ace [term 1 LEADER]: Becoming Leader. State: Replica: c32b27909a9949f48f3ea22489973ace, State: Running, Role: LEADER
I20260929 10:59:41.758142 2668 consensus_queue.cc:237] T 4e5b51c3b76a4614a81c2226b30362bc P c32b27909a9949f48f3ea22489973ace [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } }
I20260929 10:59:41.758476 2702 tablet_replica.cc:439] T 4e5b51c3b76a4614a81c2226b30362bc P c32b27909a9949f48f3ea22489973ace: TxnStatusTablet state changed. Reason: New leader c32b27909a9949f48f3ea22489973ace. Latest consensus state: current_term: 1 leader_uuid: "c32b27909a9949f48f3ea22489973ace" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } } }
I20260929 10:59:41.758571 2702 tablet_replica.cc:442] T 4e5b51c3b76a4614a81c2226b30362bc P c32b27909a9949f48f3ea22489973ace: This TxnStatusTablet replica's current role is: LEADER
I20260929 10:59:41.758651 2448 raft_consensus.cc:2454] T 4e5b51c3b76a4614a81c2226b30362bc P cff46e36bd9e400087ced7da2dbf9f0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c32b27909a9949f48f3ea22489973ace in term 1.
I20260929 10:59:41.758729 2704 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 10:59:41.758845 2704 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 10:59:41.759133 2150 catalog_manager.cc:5495] T 4e5b51c3b76a4614a81c2226b30362bc P c32b27909a9949f48f3ea22489973ace reported cstate change: term changed from 0 to 1, leader changed from <none> to c32b27909a9949f48f3ea22489973ace (127.26.120.195). New cstate: current_term: 1 leader_uuid: "c32b27909a9949f48f3ea22489973ace" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } health_report { overall_health: HEALTHY } } }
I20260929 10:59:41.770390 2661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.767926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42330 (local address 127.26.120.195:35405)
0929 10:59:41.767984 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:41.767986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:41.768012 (+ 26us) server_negotiation.cc:407] Connection header received
0929 10:59:41.768113 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:41.768114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:41.768130 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:41.768188 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:59:41.768312 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.768792 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.769569 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.769774 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.769853 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.770079 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:59:41.770175 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:59:41.770258 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:41.770310 (+ 52us) server_negotiation.cc:299] Negotiation successful
0929 10:59:41.770329 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 10:59:41.770426 2697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.767856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:35405 (local address 127.0.0.1:42330)
0929 10:59:41.767935 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:41.767964 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:41.768024 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:41.768190 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:41.768190 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:41.768191 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 10:59:41.768237 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.768239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.768880 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.768881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.769465 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.769466 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.769853 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.769854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.769927 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.769940 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 10:59:41.770245 (+ 305us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 10:59:41.770254 (+ 9us) client_negotiation.cc:769] Sending connection context
0929 10:59:41.770306 (+ 52us) client_negotiation.cc:240] Negotiation successful
0929 10:59:41.770350 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260929 10:59:41.771364 2308 raft_consensus.cc:1261] T 4e5b51c3b76a4614a81c2226b30362bc P 297c51b2364b4beb90b2d1f5282ac70a [term 1 FOLLOWER]: Refusing update from remote peer c32b27909a9949f48f3ea22489973ace: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:59:41.771543 2448 raft_consensus.cc:1261] T 4e5b51c3b76a4614a81c2226b30362bc P cff46e36bd9e400087ced7da2dbf9f0e [term 1 FOLLOWER]: Refusing update from remote peer c32b27909a9949f48f3ea22489973ace: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:59:41.771576 2702 consensus_queue.cc:1035] T 4e5b51c3b76a4614a81c2226b30362bc P c32b27909a9949f48f3ea22489973ace [LEADER]: Connected to new peer: Peer: permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:59:41.771775 2702 consensus_queue.cc:1035] T 4e5b51c3b76a4614a81c2226b30362bc P c32b27909a9949f48f3ea22489973ace [LEADER]: Connected to new peer: Peer: permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:59:41.772260 2670 tablet_replica.cc:439] T 4e5b51c3b76a4614a81c2226b30362bc P 297c51b2364b4beb90b2d1f5282ac70a: TxnStatusTablet state changed. Reason: New leader c32b27909a9949f48f3ea22489973ace. Latest consensus state: current_term: 1 leader_uuid: "c32b27909a9949f48f3ea22489973ace" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } } }
I20260929 10:59:41.772351 2670 tablet_replica.cc:442] T 4e5b51c3b76a4614a81c2226b30362bc P 297c51b2364b4beb90b2d1f5282ac70a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:59:41.772372 2681 tablet_replica.cc:439] T 4e5b51c3b76a4614a81c2226b30362bc P cff46e36bd9e400087ced7da2dbf9f0e: TxnStatusTablet state changed. Reason: New leader c32b27909a9949f48f3ea22489973ace. Latest consensus state: current_term: 1 leader_uuid: "c32b27909a9949f48f3ea22489973ace" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } } }
I20260929 10:59:41.772466 2681 tablet_replica.cc:442] T 4e5b51c3b76a4614a81c2226b30362bc P cff46e36bd9e400087ced7da2dbf9f0e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:59:41.773097 2668 tablet_replica.cc:439] T 4e5b51c3b76a4614a81c2226b30362bc P c32b27909a9949f48f3ea22489973ace: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c32b27909a9949f48f3ea22489973ace" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } } }
I20260929 10:59:41.773192 2668 tablet_replica.cc:442] T 4e5b51c3b76a4614a81c2226b30362bc P c32b27909a9949f48f3ea22489973ace: This TxnStatusTablet replica's current role is: LEADER
I20260929 10:59:41.773213 2670 tablet_replica.cc:439] T 4e5b51c3b76a4614a81c2226b30362bc P 297c51b2364b4beb90b2d1f5282ac70a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c32b27909a9949f48f3ea22489973ace" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } } }
I20260929 10:59:41.773284 2670 tablet_replica.cc:442] T 4e5b51c3b76a4614a81c2226b30362bc P 297c51b2364b4beb90b2d1f5282ac70a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:59:41.773103 2702 tablet_replica.cc:439] T 4e5b51c3b76a4614a81c2226b30362bc P c32b27909a9949f48f3ea22489973ace: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c32b27909a9949f48f3ea22489973ace" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } } }
I20260929 10:59:41.773360 2702 tablet_replica.cc:442] T 4e5b51c3b76a4614a81c2226b30362bc P c32b27909a9949f48f3ea22489973ace: This TxnStatusTablet replica's current role is: LEADER
I20260929 10:59:41.774128 2675 mvcc.cc:205] Tried to move back new op lower bound from 7334623566933712896 to 7334623566881849344. Current Snapshot: MvccSnapshot[applied={T|T < 7334623566933712896 or (T in {7334623566933712896})}]
I20260929 10:59:41.774267 2681 tablet_replica.cc:439] T 4e5b51c3b76a4614a81c2226b30362bc P cff46e36bd9e400087ced7da2dbf9f0e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c32b27909a9949f48f3ea22489973ace" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } } }
I20260929 10:59:41.774333 2681 tablet_replica.cc:442] T 4e5b51c3b76a4614a81c2226b30362bc P cff46e36bd9e400087ced7da2dbf9f0e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 10:59:41.779357 2712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.776741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:35405 (local address 127.26.120.194:57887)
0929 10:59:41.776868 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:41.776877 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:41.776917 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:41.777042 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:41.777043 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:41.777044 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 10:59:41.777080 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.777081 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.777917 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.777917 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.778559 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.778561 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.779020 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.779021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.779058 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.779070 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 10:59:41.779237 (+ 167us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 10:59:41.779239 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 10:59:41.779273 (+ 34us) client_negotiation.cc:240] Negotiation successful
0929 10:59:41.779308 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":74,"threads_started":1}
I20260929 10:59:41.779356 2661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.776884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:57887 (local address 127.26.120.195:35405)
0929 10:59:41.776920 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:41.776922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:41.776951 (+ 29us) server_negotiation.cc:407] Connection header received
0929 10:59:41.776971 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:41.776971 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:41.776981 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:41.777047 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:59:41.777119 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.777840 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.778644 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.778940 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.779011 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.779143 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:59:41.779174 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:59:41.779227 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:41.779275 (+ 48us) server_negotiation.cc:299] Negotiation successful
0929 10:59:41.779292 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 10:59:41.781329 2588 raft_consensus.cc:1261] T c8e4b7b22cb24e72ab12f65860f83056 P c32b27909a9949f48f3ea22489973ace [term 1 FOLLOWER]: Refusing update from remote peer cff46e36bd9e400087ced7da2dbf9f0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:59:41.781385 2308 raft_consensus.cc:1261] T c8e4b7b22cb24e72ab12f65860f83056 P 297c51b2364b4beb90b2d1f5282ac70a [term 1 FOLLOWER]: Refusing update from remote peer cff46e36bd9e400087ced7da2dbf9f0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:59:41.781514 2669 consensus_queue.cc:1035] T c8e4b7b22cb24e72ab12f65860f83056 P cff46e36bd9e400087ced7da2dbf9f0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:59:41.781592 2681 consensus_queue.cc:1035] T c8e4b7b22cb24e72ab12f65860f83056 P cff46e36bd9e400087ced7da2dbf9f0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:59:41.789492 2659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.786691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:54141 (local address 127.26.120.194:45525)
0929 10:59:41.786733 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:41.786735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:41.786744 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:59:41.786764 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:41.786765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:41.786772 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:41.786826 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:59:41.786954 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.787401 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.788767 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.788977 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.789065 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.789240 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:59:41.789273 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:59:41.789337 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:41.789406 (+ 69us) server_negotiation.cc:299] Negotiation successful
0929 10:59:41.789425 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 10:59:41.789499 2640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.786574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:45525 (local address 127.26.120.195:54141)
0929 10:59:41.786627 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:41.786666 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:41.786709 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:41.786830 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:41.786831 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:41.786832 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 10:59:41.786887 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.786888 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.787484 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.787485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.788686 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.788688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.789067 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.789067 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.789129 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.789150 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 10:59:41.789357 (+ 207us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 10:59:41.789358 (+ 1us) client_negotiation.cc:769] Sending connection context
0929 10:59:41.789402 (+ 44us) client_negotiation.cc:240] Negotiation successful
0929 10:59:41.789438 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260929 10:59:41.798142 2151 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:59:41.798236 2151 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:59:41.798894 2151 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5b33e54e771e4830afa648431a0880a2: Sending DeleteTablet for 3 replicas of tablet c8e4b7b22cb24e72ab12f65860f83056
I20260929 10:59:41.799110 2288 tablet_service.cc:1514] Processing DeleteTablet for tablet c8e4b7b22cb24e72ab12f65860f83056 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:41 UTC) from {username='oryx', principal='oryx/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:33744
I20260929 10:59:41.799208 2687 tablet_replica.cc:330] T c8e4b7b22cb24e72ab12f65860f83056 P 297c51b2364b4beb90b2d1f5282ac70a: stopping tablet replica
I20260929 10:59:41.799221 2428 tablet_service.cc:1514] Processing DeleteTablet for tablet c8e4b7b22cb24e72ab12f65860f83056 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:41 UTC) from {username='oryx', principal='oryx/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:41072
I20260929 10:59:41.799258 2687 raft_consensus.cc:2229] T c8e4b7b22cb24e72ab12f65860f83056 P 297c51b2364b4beb90b2d1f5282ac70a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:59:41.799297 2687 raft_consensus.cc:2258] T c8e4b7b22cb24e72ab12f65860f83056 P 297c51b2364b4beb90b2d1f5282ac70a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:41.799366 2686 tablet_replica.cc:330] T c8e4b7b22cb24e72ab12f65860f83056 P cff46e36bd9e400087ced7da2dbf9f0e: stopping tablet replica
I20260929 10:59:41.799376 2568 tablet_service.cc:1514] Processing DeleteTablet for tablet c8e4b7b22cb24e72ab12f65860f83056 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:41 UTC) from {username='oryx', principal='oryx/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:42314
I20260929 10:59:41.799418 2686 raft_consensus.cc:2229] T c8e4b7b22cb24e72ab12f65860f83056 P cff46e36bd9e400087ced7da2dbf9f0e [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:59:41.799465 2686 raft_consensus.cc:2258] T c8e4b7b22cb24e72ab12f65860f83056 P cff46e36bd9e400087ced7da2dbf9f0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:41.799471 2685 tablet_replica.cc:330] T c8e4b7b22cb24e72ab12f65860f83056 P c32b27909a9949f48f3ea22489973ace: stopping tablet replica
I20260929 10:59:41.799616 2685 raft_consensus.cc:2229] T c8e4b7b22cb24e72ab12f65860f83056 P c32b27909a9949f48f3ea22489973ace [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:59:41.799697 2685 raft_consensus.cc:2258] T c8e4b7b22cb24e72ab12f65860f83056 P c32b27909a9949f48f3ea22489973ace [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:41.799904 2686 ts_tablet_manager.cc:1905] T c8e4b7b22cb24e72ab12f65860f83056 P cff46e36bd9e400087ced7da2dbf9f0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:41.799981 2685 ts_tablet_manager.cc:1905] T c8e4b7b22cb24e72ab12f65860f83056 P c32b27909a9949f48f3ea22489973ace: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:41.801239 2687 ts_tablet_manager.cc:1905] T c8e4b7b22cb24e72ab12f65860f83056 P 297c51b2364b4beb90b2d1f5282ac70a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:41.801524 2685 ts_tablet_manager.cc:1918] T c8e4b7b22cb24e72ab12f65860f83056 P c32b27909a9949f48f3ea22489973ace: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:59:41.801573 2685 log.cc:1196] T c8e4b7b22cb24e72ab12f65860f83056 P c32b27909a9949f48f3ea22489973ace: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-2/wal/wals/c8e4b7b22cb24e72ab12f65860f83056
I20260929 10:59:41.801783 2685 ts_tablet_manager.cc:1939] T c8e4b7b22cb24e72ab12f65860f83056 P c32b27909a9949f48f3ea22489973ace: Deleting consensus metadata
I20260929 10:59:41.802215 2686 ts_tablet_manager.cc:1918] T c8e4b7b22cb24e72ab12f65860f83056 P cff46e36bd9e400087ced7da2dbf9f0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:59:41.802273 2686 log.cc:1196] T c8e4b7b22cb24e72ab12f65860f83056 P cff46e36bd9e400087ced7da2dbf9f0e: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-1/wal/wals/c8e4b7b22cb24e72ab12f65860f83056
I20260929 10:59:41.802498 2686 ts_tablet_manager.cc:1939] T c8e4b7b22cb24e72ab12f65860f83056 P cff46e36bd9e400087ced7da2dbf9f0e: Deleting consensus metadata
I20260929 10:59:41.803061 2687 ts_tablet_manager.cc:1918] T c8e4b7b22cb24e72ab12f65860f83056 P 297c51b2364b4beb90b2d1f5282ac70a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:59:41.803129 2687 log.cc:1196] T c8e4b7b22cb24e72ab12f65860f83056 P 297c51b2364b4beb90b2d1f5282ac70a: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-0/wal/wals/c8e4b7b22cb24e72ab12f65860f83056
I20260929 10:59:41.803349 2687 ts_tablet_manager.cc:1939] T c8e4b7b22cb24e72ab12f65860f83056 P 297c51b2364b4beb90b2d1f5282ac70a: Deleting consensus metadata
I20260929 10:59:41.803591 2138 catalog_manager.cc:4841] TS cff46e36bd9e400087ced7da2dbf9f0e (127.26.120.194:45525): tablet c8e4b7b22cb24e72ab12f65860f83056 (table test-table [id=aea4ba9562144f6b81e8991f53217f95]) successfully deleted
I20260929 10:59:41.803653 2135 catalog_manager.cc:4841] TS 297c51b2364b4beb90b2d1f5282ac70a (127.26.120.193:46459): tablet c8e4b7b22cb24e72ab12f65860f83056 (table test-table [id=aea4ba9562144f6b81e8991f53217f95]) successfully deleted
I20260929 10:59:41.804229 2137 catalog_manager.cc:4841] TS c32b27909a9949f48f3ea22489973ace (127.26.120.195:35405): tablet c8e4b7b22cb24e72ab12f65860f83056 (table test-table [id=aea4ba9562144f6b81e8991f53217f95]) successfully deleted
Sep 29 10:59:41 dist-test-slave-2lzq krb5kdc[2106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 10:59:41 dist-test-slave-2lzq krb5kdc[2106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:41 dist-test-slave-2lzq krb5kdc[2106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
W20260929 10:59:41.825393 2201 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:41.819392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59660 (local address 127.26.120.254:43649)
0929 10:59:41.819472 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:41.819475 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:41.819583 (+ 108us) server_negotiation.cc:407] Connection header received
0929 10:59:41.819622 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:41.819623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:41.819636 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:41.819712 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:41.820272 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.821055 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.821793 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.822201 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.822277 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.825322 (+ 3045us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59660: BlockingRecv error: recv got EOF from 127.0.0.1:59660 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 29 10:59:41 dist-test-slave-2lzq krb5kdc[2106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM
I20260929 10:59:41.833493 2201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.826246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59674 (local address 127.26.120.254:43649)
0929 10:59:41.826286 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:41.826288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:41.826319 (+ 31us) server_negotiation.cc:407] Connection header received
0929 10:59:41.826392 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:41.826393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:41.826401 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:41.826445 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:41.827027 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.827476 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.828483 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.828671 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.828721 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.831002 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:41.831003 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:41.831004 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:41.831011 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:41.832375 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:41.832729 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:41.832730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:41.832731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:41.832790 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:41.832987 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:41.832988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:41.832988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:41.833102 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:41.833201 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:41.833320 (+ 119us) server_negotiation.cc:299] Negotiation successful
0929 10:59:41.833390 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 10:59:41.835026 2151 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59674:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:59:41.835175 2151 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:59:41.836575 2428 tablet_service.cc:1467] Processing CreateTablet for tablet ce5df6291af2431d885339777b03bbb9 (DEFAULT_TABLE table=test-table [id=c12a95bebb7a41dfb2b4d1bce86029a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:41.836584 2288 tablet_service.cc:1467] Processing CreateTablet for tablet ce5df6291af2431d885339777b03bbb9 (DEFAULT_TABLE table=test-table [id=c12a95bebb7a41dfb2b4d1bce86029a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:41.836733 2428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce5df6291af2431d885339777b03bbb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:41.836735 2288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce5df6291af2431d885339777b03bbb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:41.837827 2663 tablet_bootstrap.cc:492] T ce5df6291af2431d885339777b03bbb9 P cff46e36bd9e400087ced7da2dbf9f0e: Bootstrap starting.
I20260929 10:59:41.838227 2663 tablet_bootstrap.cc:654] T ce5df6291af2431d885339777b03bbb9 P cff46e36bd9e400087ced7da2dbf9f0e: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:41.838325 2568 tablet_service.cc:1467] Processing CreateTablet for tablet ce5df6291af2431d885339777b03bbb9 (DEFAULT_TABLE table=test-table [id=c12a95bebb7a41dfb2b4d1bce86029a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:41.838440 2568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce5df6291af2431d885339777b03bbb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:41.838742 2663 tablet_bootstrap.cc:492] T ce5df6291af2431d885339777b03bbb9 P cff46e36bd9e400087ced7da2dbf9f0e: No bootstrap required, opened a new log
I20260929 10:59:41.838809 2663 ts_tablet_manager.cc:1397] T ce5df6291af2431d885339777b03bbb9 P cff46e36bd9e400087ced7da2dbf9f0e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 10:59:41.838967 2663 raft_consensus.cc:348] T ce5df6291af2431d885339777b03bbb9 P cff46e36bd9e400087ced7da2dbf9f0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } }
I20260929 10:59:41.839066 2663 raft_consensus.cc:374] T ce5df6291af2431d885339777b03bbb9 P cff46e36bd9e400087ced7da2dbf9f0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:41.839089 2663 raft_consensus.cc:729] T ce5df6291af2431d885339777b03bbb9 P cff46e36bd9e400087ced7da2dbf9f0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cff46e36bd9e400087ced7da2dbf9f0e, State: Initialized, Role: FOLLOWER
I20260929 10:59:41.839136 2663 consensus_queue.cc:260] T ce5df6291af2431d885339777b03bbb9 P cff46e36bd9e400087ced7da2dbf9f0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } }
I20260929 10:59:41.839231 2663 ts_tablet_manager.cc:1428] T ce5df6291af2431d885339777b03bbb9 P cff46e36bd9e400087ced7da2dbf9f0e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:41.839775 2664 tablet_bootstrap.cc:492] T ce5df6291af2431d885339777b03bbb9 P c32b27909a9949f48f3ea22489973ace: Bootstrap starting.
I20260929 10:59:41.840096 2664 tablet_bootstrap.cc:654] T ce5df6291af2431d885339777b03bbb9 P c32b27909a9949f48f3ea22489973ace: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:41.840559 2662 tablet_bootstrap.cc:492] T ce5df6291af2431d885339777b03bbb9 P 297c51b2364b4beb90b2d1f5282ac70a: Bootstrap starting.
I20260929 10:59:41.840755 2664 tablet_bootstrap.cc:492] T ce5df6291af2431d885339777b03bbb9 P c32b27909a9949f48f3ea22489973ace: No bootstrap required, opened a new log
I20260929 10:59:41.840797 2664 ts_tablet_manager.cc:1397] T ce5df6291af2431d885339777b03bbb9 P c32b27909a9949f48f3ea22489973ace: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 10:59:41.840911 2662 tablet_bootstrap.cc:654] T ce5df6291af2431d885339777b03bbb9 P 297c51b2364b4beb90b2d1f5282ac70a: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:41.840950 2664 raft_consensus.cc:348] T ce5df6291af2431d885339777b03bbb9 P c32b27909a9949f48f3ea22489973ace [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } }
I20260929 10:59:41.841010 2664 raft_consensus.cc:374] T ce5df6291af2431d885339777b03bbb9 P c32b27909a9949f48f3ea22489973ace [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:41.841027 2664 raft_consensus.cc:729] T ce5df6291af2431d885339777b03bbb9 P c32b27909a9949f48f3ea22489973ace [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c32b27909a9949f48f3ea22489973ace, State: Initialized, Role: FOLLOWER
I20260929 10:59:41.841084 2664 consensus_queue.cc:260] T ce5df6291af2431d885339777b03bbb9 P c32b27909a9949f48f3ea22489973ace [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } }
I20260929 10:59:41.841168 2664 ts_tablet_manager.cc:1428] T ce5df6291af2431d885339777b03bbb9 P c32b27909a9949f48f3ea22489973ace: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260929 10:59:41.841511 2662 tablet_bootstrap.cc:492] T ce5df6291af2431d885339777b03bbb9 P 297c51b2364b4beb90b2d1f5282ac70a: No bootstrap required, opened a new log
I20260929 10:59:41.841583 2662 ts_tablet_manager.cc:1397] T ce5df6291af2431d885339777b03bbb9 P 297c51b2364b4beb90b2d1f5282ac70a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 10:59:41.841735 2662 raft_consensus.cc:348] T ce5df6291af2431d885339777b03bbb9 P 297c51b2364b4beb90b2d1f5282ac70a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } }
I20260929 10:59:41.841813 2662 raft_consensus.cc:374] T ce5df6291af2431d885339777b03bbb9 P 297c51b2364b4beb90b2d1f5282ac70a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:41.841837 2662 raft_consensus.cc:729] T ce5df6291af2431d885339777b03bbb9 P 297c51b2364b4beb90b2d1f5282ac70a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297c51b2364b4beb90b2d1f5282ac70a, State: Initialized, Role: FOLLOWER
I20260929 10:59:41.841897 2662 consensus_queue.cc:260] T ce5df6291af2431d885339777b03bbb9 P 297c51b2364b4beb90b2d1f5282ac70a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } }
I20260929 10:59:41.842007 2662 ts_tablet_manager.cc:1428] T ce5df6291af2431d885339777b03bbb9 P 297c51b2364b4beb90b2d1f5282ac70a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:41.854199 2670 raft_consensus.cc:482] T ce5df6291af2431d885339777b03bbb9 P 297c51b2364b4beb90b2d1f5282ac70a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:59:41.854291 2670 raft_consensus.cc:504] T ce5df6291af2431d885339777b03bbb9 P 297c51b2364b4beb90b2d1f5282ac70a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } }
I20260929 10:59:41.854631 2670 leader_election.cc:288] T ce5df6291af2431d885339777b03bbb9 P 297c51b2364b4beb90b2d1f5282ac70a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cff46e36bd9e400087ced7da2dbf9f0e (127.26.120.194:45525), c32b27909a9949f48f3ea22489973ace (127.26.120.195:35405)
I20260929 10:59:41.858534 2659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.854888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:40885 (local address 127.26.120.194:45525)
0929 10:59:41.854936 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:41.854938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:41.855066 (+ 128us) server_negotiation.cc:407] Connection header received
0929 10:59:41.855139 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:41.855140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:41.855154 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:41.855200 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:41.855408 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.856001 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.857533 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.858148 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.858228 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.858239 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:41.858419 (+ 180us) server_negotiation.cc:299] Negotiation successful
0929 10:59:41.858463 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 10:59:41.858543 2733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.854788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:45525 (local address 127.26.120.193:40885)
0929 10:59:41.854982 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:41.854992 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:41.855083 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:41.855250 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:41.855251 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:41.855253 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:41.855338 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.855340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.856086 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.856087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.857425 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.857428 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.858250 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.858251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.858332 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.858337 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 10:59:41.858397 (+ 60us) client_negotiation.cc:240] Negotiation successful
0929 10:59:41.858438 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":58,"threads_started":1}
I20260929 10:59:41.858855 2448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce5df6291af2431d885339777b03bbb9" candidate_uuid: "297c51b2364b4beb90b2d1f5282ac70a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" is_pre_election: true
I20260929 10:59:41.858958 2448 raft_consensus.cc:2454] T ce5df6291af2431d885339777b03bbb9 P cff46e36bd9e400087ced7da2dbf9f0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 297c51b2364b4beb90b2d1f5282ac70a in term 0.
I20260929 10:59:41.858999 2661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.854907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:38907 (local address 127.26.120.195:35405)
0929 10:59:41.854954 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:41.854956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:41.855074 (+ 118us) server_negotiation.cc:407] Connection header received
0929 10:59:41.855189 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:41.855190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:41.855201 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:41.855253 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 10:59:41.855620 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.856326 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.857848 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.858355 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.858416 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.858424 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:41.858897 (+ 473us) server_negotiation.cc:299] Negotiation successful
0929 10:59:41.858935 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 10:59:41.859205 2734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.854794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:35405 (local address 127.26.120.193:38907)
0929 10:59:41.854999 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:41.855009 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:41.855102 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:41.855258 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:41.855259 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:41.855261 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 10:59:41.855356 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.855528 (+ 172us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.856431 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.856432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.857763 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:41.857764 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.858454 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.858454 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:41.858521 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.858524 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 10:59:41.859087 (+ 563us) client_negotiation.cc:240] Negotiation successful
0929 10:59:41.859134 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":71,"threads_started":1}
I20260929 10:59:41.859201 2242 leader_election.cc:302] T ce5df6291af2431d885339777b03bbb9 P 297c51b2364b4beb90b2d1f5282ac70a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297c51b2364b4beb90b2d1f5282ac70a, cff46e36bd9e400087ced7da2dbf9f0e; no voters:
I20260929 10:59:41.859498 2588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce5df6291af2431d885339777b03bbb9" candidate_uuid: "297c51b2364b4beb90b2d1f5282ac70a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c32b27909a9949f48f3ea22489973ace" is_pre_election: true
I20260929 10:59:41.859529 2670 raft_consensus.cc:2781] T ce5df6291af2431d885339777b03bbb9 P 297c51b2364b4beb90b2d1f5282ac70a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:59:41.859577 2670 raft_consensus.cc:482] T ce5df6291af2431d885339777b03bbb9 P 297c51b2364b4beb90b2d1f5282ac70a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:59:41.859586 2588 raft_consensus.cc:2454] T ce5df6291af2431d885339777b03bbb9 P c32b27909a9949f48f3ea22489973ace [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 297c51b2364b4beb90b2d1f5282ac70a in term 0.
I20260929 10:59:41.859604 2670 raft_consensus.cc:3037] T ce5df6291af2431d885339777b03bbb9 P 297c51b2364b4beb90b2d1f5282ac70a [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:41.860486 2670 raft_consensus.cc:504] T ce5df6291af2431d885339777b03bbb9 P 297c51b2364b4beb90b2d1f5282ac70a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } }
I20260929 10:59:41.860628 2670 leader_election.cc:288] T ce5df6291af2431d885339777b03bbb9 P 297c51b2364b4beb90b2d1f5282ac70a [CANDIDATE]: Term 1 election: Requested vote from peers cff46e36bd9e400087ced7da2dbf9f0e (127.26.120.194:45525), c32b27909a9949f48f3ea22489973ace (127.26.120.195:35405)
I20260929 10:59:41.860805 2588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce5df6291af2431d885339777b03bbb9" candidate_uuid: "297c51b2364b4beb90b2d1f5282ac70a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c32b27909a9949f48f3ea22489973ace"
I20260929 10:59:41.860874 2588 raft_consensus.cc:3037] T ce5df6291af2431d885339777b03bbb9 P c32b27909a9949f48f3ea22489973ace [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:41.860853 2448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce5df6291af2431d885339777b03bbb9" candidate_uuid: "297c51b2364b4beb90b2d1f5282ac70a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cff46e36bd9e400087ced7da2dbf9f0e"
I20260929 10:59:41.860917 2448 raft_consensus.cc:3037] T ce5df6291af2431d885339777b03bbb9 P cff46e36bd9e400087ced7da2dbf9f0e [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:41.861625 2588 raft_consensus.cc:2454] T ce5df6291af2431d885339777b03bbb9 P c32b27909a9949f48f3ea22489973ace [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 297c51b2364b4beb90b2d1f5282ac70a in term 1.
I20260929 10:59:41.861663 2448 raft_consensus.cc:2454] T ce5df6291af2431d885339777b03bbb9 P cff46e36bd9e400087ced7da2dbf9f0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 297c51b2364b4beb90b2d1f5282ac70a in term 1.
I20260929 10:59:41.861830 2241 leader_election.cc:302] T ce5df6291af2431d885339777b03bbb9 P 297c51b2364b4beb90b2d1f5282ac70a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297c51b2364b4beb90b2d1f5282ac70a, c32b27909a9949f48f3ea22489973ace; no voters:
I20260929 10:59:41.861914 2670 raft_consensus.cc:2781] T ce5df6291af2431d885339777b03bbb9 P 297c51b2364b4beb90b2d1f5282ac70a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:59:41.862123 2670 raft_consensus.cc:686] T ce5df6291af2431d885339777b03bbb9 P 297c51b2364b4beb90b2d1f5282ac70a [term 1 LEADER]: Becoming Leader. State: Replica: 297c51b2364b4beb90b2d1f5282ac70a, State: Running, Role: LEADER
I20260929 10:59:41.862200 2670 consensus_queue.cc:237] T ce5df6291af2431d885339777b03bbb9 P 297c51b2364b4beb90b2d1f5282ac70a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } }
I20260929 10:59:41.862852 2151 catalog_manager.cc:5495] T ce5df6291af2431d885339777b03bbb9 P 297c51b2364b4beb90b2d1f5282ac70a reported cstate change: term changed from 0 to 1, leader changed from <none> to 297c51b2364b4beb90b2d1f5282ac70a (127.26.120.193). New cstate: current_term: 1 leader_uuid: "297c51b2364b4beb90b2d1f5282ac70a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } health_report { overall_health: UNKNOWN } } }
I20260929 10:59:41.871599 2660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:41.869248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33756 (local address 127.26.120.193:46459)
0929 10:59:41.869310 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:41.869312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:41.869320 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:59:41.869351 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:41.869352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:41.869373 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:41.869478 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:59:41.869600 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.870109 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.870832 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:41.871042 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:41.871118 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:41.871274 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:59:41.871380 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:59:41.871466 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:41.871505 (+ 39us) server_negotiation.cc:299] Negotiation successful
0929 10:59:41.871526 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 10:59:41.872552 2448 raft_consensus.cc:1261] T ce5df6291af2431d885339777b03bbb9 P cff46e36bd9e400087ced7da2dbf9f0e [term 1 FOLLOWER]: Refusing update from remote peer 297c51b2364b4beb90b2d1f5282ac70a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:59:41.872552 2588 raft_consensus.cc:1261] T ce5df6291af2431d885339777b03bbb9 P c32b27909a9949f48f3ea22489973ace [term 1 FOLLOWER]: Refusing update from remote peer 297c51b2364b4beb90b2d1f5282ac70a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:59:41.872871 2735 consensus_queue.cc:1035] T ce5df6291af2431d885339777b03bbb9 P 297c51b2364b4beb90b2d1f5282ac70a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:59:41.873028 2670 consensus_queue.cc:1035] T ce5df6291af2431d885339777b03bbb9 P 297c51b2364b4beb90b2d1f5282ac70a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:59:41.876255 2151 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:59:41.876343 2151 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:59:41.876940 2151 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5b33e54e771e4830afa648431a0880a2: Sending DeleteTablet for 3 replicas of tablet ce5df6291af2431d885339777b03bbb9
I20260929 10:59:41.877192 2288 tablet_service.cc:1514] Processing DeleteTablet for tablet ce5df6291af2431d885339777b03bbb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:41 UTC) from {username='oryx', principal='oryx/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:33744
I20260929 10:59:41.877286 2568 tablet_service.cc:1514] Processing DeleteTablet for tablet ce5df6291af2431d885339777b03bbb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:41 UTC) from {username='oryx', principal='oryx/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:42314
I20260929 10:59:41.877398 2687 tablet_replica.cc:330] T ce5df6291af2431d885339777b03bbb9 P 297c51b2364b4beb90b2d1f5282ac70a: stopping tablet replica
I20260929 10:59:41.877455 2687 raft_consensus.cc:2229] T ce5df6291af2431d885339777b03bbb9 P 297c51b2364b4beb90b2d1f5282ac70a [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:59:41.877540 2687 raft_consensus.cc:2258] T ce5df6291af2431d885339777b03bbb9 P 297c51b2364b4beb90b2d1f5282ac70a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:41.877615 2685 tablet_replica.cc:330] T ce5df6291af2431d885339777b03bbb9 P c32b27909a9949f48f3ea22489973ace: stopping tablet replica
I20260929 10:59:41.877667 2685 raft_consensus.cc:2229] T ce5df6291af2431d885339777b03bbb9 P c32b27909a9949f48f3ea22489973ace [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:59:41.877713 2685 raft_consensus.cc:2258] T ce5df6291af2431d885339777b03bbb9 P c32b27909a9949f48f3ea22489973ace [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:41.877988 2687 ts_tablet_manager.cc:1905] T ce5df6291af2431d885339777b03bbb9 P 297c51b2364b4beb90b2d1f5282ac70a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:41.878029 2428 tablet_service.cc:1514] Processing DeleteTablet for tablet ce5df6291af2431d885339777b03bbb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:41 UTC) from {username='oryx', principal='oryx/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:41072
I20260929 10:59:41.878189 2686 tablet_replica.cc:330] T ce5df6291af2431d885339777b03bbb9 P cff46e36bd9e400087ced7da2dbf9f0e: stopping tablet replica
I20260929 10:59:41.878230 2686 raft_consensus.cc:2229] T ce5df6291af2431d885339777b03bbb9 P cff46e36bd9e400087ced7da2dbf9f0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:59:41.878265 2686 raft_consensus.cc:2258] T ce5df6291af2431d885339777b03bbb9 P cff46e36bd9e400087ced7da2dbf9f0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:41.878561 2151 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59674:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 10:59:41.878688 2151 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 10:59:41.879817 2288 tablet_service.cc:1467] Processing CreateTablet for tablet f1c7f48759624ab8a095ad653da47628 (DEFAULT_TABLE table=test-table [id=16a0f197caf948f6882855dc53324aef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:41.879923 2568 tablet_service.cc:1467] Processing CreateTablet for tablet f1c7f48759624ab8a095ad653da47628 (DEFAULT_TABLE table=test-table [id=16a0f197caf948f6882855dc53324aef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:41.880050 2568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1c7f48759624ab8a095ad653da47628. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:41.881384 2664 tablet_bootstrap.cc:492] T f1c7f48759624ab8a095ad653da47628 P c32b27909a9949f48f3ea22489973ace: Bootstrap starting.
I20260929 10:59:41.881645 2428 tablet_service.cc:1467] Processing CreateTablet for tablet f1c7f48759624ab8a095ad653da47628 (DEFAULT_TABLE table=test-table [id=16a0f197caf948f6882855dc53324aef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 10:59:41.881753 2428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1c7f48759624ab8a095ad653da47628. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:41.881800 2664 tablet_bootstrap.cc:654] T f1c7f48759624ab8a095ad653da47628 P c32b27909a9949f48f3ea22489973ace: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:41.882462 2664 tablet_bootstrap.cc:492] T f1c7f48759624ab8a095ad653da47628 P c32b27909a9949f48f3ea22489973ace: No bootstrap required, opened a new log
I20260929 10:59:41.882519 2664 ts_tablet_manager.cc:1397] T f1c7f48759624ab8a095ad653da47628 P c32b27909a9949f48f3ea22489973ace: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 10:59:41.882649 2664 raft_consensus.cc:348] T f1c7f48759624ab8a095ad653da47628 P c32b27909a9949f48f3ea22489973ace [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } }
I20260929 10:59:41.882709 2664 raft_consensus.cc:374] T f1c7f48759624ab8a095ad653da47628 P c32b27909a9949f48f3ea22489973ace [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:41.882730 2664 raft_consensus.cc:729] T f1c7f48759624ab8a095ad653da47628 P c32b27909a9949f48f3ea22489973ace [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c32b27909a9949f48f3ea22489973ace, State: Initialized, Role: FOLLOWER
I20260929 10:59:41.882779 2664 consensus_queue.cc:260] T f1c7f48759624ab8a095ad653da47628 P c32b27909a9949f48f3ea22489973ace [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } }
I20260929 10:59:41.882862 2664 ts_tablet_manager.cc:1428] T f1c7f48759624ab8a095ad653da47628 P c32b27909a9949f48f3ea22489973ace: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:41.883001 2663 tablet_bootstrap.cc:492] T f1c7f48759624ab8a095ad653da47628 P cff46e36bd9e400087ced7da2dbf9f0e: Bootstrap starting.
I20260929 10:59:41.883297 2663 tablet_bootstrap.cc:654] T f1c7f48759624ab8a095ad653da47628 P cff46e36bd9e400087ced7da2dbf9f0e: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:41.883688 2687 ts_tablet_manager.cc:1918] T ce5df6291af2431d885339777b03bbb9 P 297c51b2364b4beb90b2d1f5282ac70a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:59:41.883697 2288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1c7f48759624ab8a095ad653da47628. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:41.883754 2687 log.cc:1196] T ce5df6291af2431d885339777b03bbb9 P 297c51b2364b4beb90b2d1f5282ac70a: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-0/wal/wals/ce5df6291af2431d885339777b03bbb9
I20260929 10:59:41.883968 2687 ts_tablet_manager.cc:1939] T ce5df6291af2431d885339777b03bbb9 P 297c51b2364b4beb90b2d1f5282ac70a: Deleting consensus metadata
I20260929 10:59:41.884258 2135 catalog_manager.cc:4841] TS 297c51b2364b4beb90b2d1f5282ac70a (127.26.120.193:46459): tablet ce5df6291af2431d885339777b03bbb9 (table test-table [id=c12a95bebb7a41dfb2b4d1bce86029a5]) successfully deleted
I20260929 10:59:41.884753 2685 ts_tablet_manager.cc:1905] T ce5df6291af2431d885339777b03bbb9 P c32b27909a9949f48f3ea22489973ace: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:41.884846 2662 tablet_bootstrap.cc:492] T f1c7f48759624ab8a095ad653da47628 P 297c51b2364b4beb90b2d1f5282ac70a: Bootstrap starting.
I20260929 10:59:41.885205 2662 tablet_bootstrap.cc:654] T f1c7f48759624ab8a095ad653da47628 P 297c51b2364b4beb90b2d1f5282ac70a: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:41.886113 2662 tablet_bootstrap.cc:492] T f1c7f48759624ab8a095ad653da47628 P 297c51b2364b4beb90b2d1f5282ac70a: No bootstrap required, opened a new log
I20260929 10:59:41.886154 2685 ts_tablet_manager.cc:1918] T ce5df6291af2431d885339777b03bbb9 P c32b27909a9949f48f3ea22489973ace: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:59:41.886164 2662 ts_tablet_manager.cc:1397] T f1c7f48759624ab8a095ad653da47628 P 297c51b2364b4beb90b2d1f5282ac70a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 10:59:41.886204 2685 log.cc:1196] T ce5df6291af2431d885339777b03bbb9 P c32b27909a9949f48f3ea22489973ace: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-2/wal/wals/ce5df6291af2431d885339777b03bbb9
I20260929 10:59:41.886298 2662 raft_consensus.cc:348] T f1c7f48759624ab8a095ad653da47628 P 297c51b2364b4beb90b2d1f5282ac70a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } }
I20260929 10:59:41.886380 2662 raft_consensus.cc:374] T f1c7f48759624ab8a095ad653da47628 P 297c51b2364b4beb90b2d1f5282ac70a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:41.886399 2662 raft_consensus.cc:729] T f1c7f48759624ab8a095ad653da47628 P 297c51b2364b4beb90b2d1f5282ac70a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297c51b2364b4beb90b2d1f5282ac70a, State: Initialized, Role: FOLLOWER
I20260929 10:59:41.886437 2685 ts_tablet_manager.cc:1939] T ce5df6291af2431d885339777b03bbb9 P c32b27909a9949f48f3ea22489973ace: Deleting consensus metadata
I20260929 10:59:41.886435 2662 consensus_queue.cc:260] T f1c7f48759624ab8a095ad653da47628 P 297c51b2364b4beb90b2d1f5282ac70a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } }
I20260929 10:59:41.886526 2662 ts_tablet_manager.cc:1428] T f1c7f48759624ab8a095ad653da47628 P 297c51b2364b4beb90b2d1f5282ac70a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:41.887233 2137 catalog_manager.cc:4841] TS c32b27909a9949f48f3ea22489973ace (127.26.120.195:35405): tablet ce5df6291af2431d885339777b03bbb9 (table test-table [id=c12a95bebb7a41dfb2b4d1bce86029a5]) successfully deleted
I20260929 10:59:41.887532 2686 ts_tablet_manager.cc:1905] T ce5df6291af2431d885339777b03bbb9 P cff46e36bd9e400087ced7da2dbf9f0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:41.889206 2686 ts_tablet_manager.cc:1918] T ce5df6291af2431d885339777b03bbb9 P cff46e36bd9e400087ced7da2dbf9f0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 10:59:41.889261 2686 log.cc:1196] T ce5df6291af2431d885339777b03bbb9 P cff46e36bd9e400087ced7da2dbf9f0e: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-1/wal/wals/ce5df6291af2431d885339777b03bbb9
I20260929 10:59:41.889457 2686 ts_tablet_manager.cc:1939] T ce5df6291af2431d885339777b03bbb9 P cff46e36bd9e400087ced7da2dbf9f0e: Deleting consensus metadata
I20260929 10:59:41.889758 2138 catalog_manager.cc:4841] TS cff46e36bd9e400087ced7da2dbf9f0e (127.26.120.194:45525): tablet ce5df6291af2431d885339777b03bbb9 (table test-table [id=c12a95bebb7a41dfb2b4d1bce86029a5]) successfully deleted
I20260929 10:59:41.894380 2663 tablet_bootstrap.cc:492] T f1c7f48759624ab8a095ad653da47628 P cff46e36bd9e400087ced7da2dbf9f0e: No bootstrap required, opened a new log
I20260929 10:59:41.894428 2663 ts_tablet_manager.cc:1397] T f1c7f48759624ab8a095ad653da47628 P cff46e36bd9e400087ced7da2dbf9f0e: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20260929 10:59:41.894541 2663 raft_consensus.cc:348] T f1c7f48759624ab8a095ad653da47628 P cff46e36bd9e400087ced7da2dbf9f0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } }
I20260929 10:59:41.894603 2663 raft_consensus.cc:374] T f1c7f48759624ab8a095ad653da47628 P cff46e36bd9e400087ced7da2dbf9f0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 10:59:41.894622 2663 raft_consensus.cc:729] T f1c7f48759624ab8a095ad653da47628 P cff46e36bd9e400087ced7da2dbf9f0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cff46e36bd9e400087ced7da2dbf9f0e, State: Initialized, Role: FOLLOWER
I20260929 10:59:41.894659 2663 consensus_queue.cc:260] T f1c7f48759624ab8a095ad653da47628 P cff46e36bd9e400087ced7da2dbf9f0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } }
I20260929 10:59:41.894737 2663 ts_tablet_manager.cc:1428] T f1c7f48759624ab8a095ad653da47628 P cff46e36bd9e400087ced7da2dbf9f0e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:42.032501 2668 raft_consensus.cc:482] T f1c7f48759624ab8a095ad653da47628 P c32b27909a9949f48f3ea22489973ace [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 10:59:42.032609 2668 raft_consensus.cc:504] T f1c7f48759624ab8a095ad653da47628 P c32b27909a9949f48f3ea22489973ace [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } }
I20260929 10:59:42.032812 2668 leader_election.cc:288] T f1c7f48759624ab8a095ad653da47628 P c32b27909a9949f48f3ea22489973ace [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297c51b2364b4beb90b2d1f5282ac70a (127.26.120.193:46459), cff46e36bd9e400087ced7da2dbf9f0e (127.26.120.194:45525)
I20260929 10:59:42.033056 2448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1c7f48759624ab8a095ad653da47628" candidate_uuid: "c32b27909a9949f48f3ea22489973ace" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" is_pre_election: true
I20260929 10:59:42.033089 2308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1c7f48759624ab8a095ad653da47628" candidate_uuid: "c32b27909a9949f48f3ea22489973ace" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297c51b2364b4beb90b2d1f5282ac70a" is_pre_election: true
I20260929 10:59:42.033167 2308 raft_consensus.cc:2454] T f1c7f48759624ab8a095ad653da47628 P 297c51b2364b4beb90b2d1f5282ac70a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c32b27909a9949f48f3ea22489973ace in term 0.
I20260929 10:59:42.033185 2448 raft_consensus.cc:2454] T f1c7f48759624ab8a095ad653da47628 P cff46e36bd9e400087ced7da2dbf9f0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c32b27909a9949f48f3ea22489973ace in term 0.
I20260929 10:59:42.033355 2522 leader_election.cc:302] T f1c7f48759624ab8a095ad653da47628 P c32b27909a9949f48f3ea22489973ace [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c32b27909a9949f48f3ea22489973ace, cff46e36bd9e400087ced7da2dbf9f0e; no voters:
I20260929 10:59:42.033488 2668 raft_consensus.cc:2781] T f1c7f48759624ab8a095ad653da47628 P c32b27909a9949f48f3ea22489973ace [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 10:59:42.033538 2668 raft_consensus.cc:482] T f1c7f48759624ab8a095ad653da47628 P c32b27909a9949f48f3ea22489973ace [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 10:59:42.033556 2668 raft_consensus.cc:3037] T f1c7f48759624ab8a095ad653da47628 P c32b27909a9949f48f3ea22489973ace [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:42.034250 2668 raft_consensus.cc:504] T f1c7f48759624ab8a095ad653da47628 P c32b27909a9949f48f3ea22489973ace [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } }
I20260929 10:59:42.034406 2668 leader_election.cc:288] T f1c7f48759624ab8a095ad653da47628 P c32b27909a9949f48f3ea22489973ace [CANDIDATE]: Term 1 election: Requested vote from peers 297c51b2364b4beb90b2d1f5282ac70a (127.26.120.193:46459), cff46e36bd9e400087ced7da2dbf9f0e (127.26.120.194:45525)
I20260929 10:59:42.034565 2308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1c7f48759624ab8a095ad653da47628" candidate_uuid: "c32b27909a9949f48f3ea22489973ace" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297c51b2364b4beb90b2d1f5282ac70a"
I20260929 10:59:42.034577 2448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1c7f48759624ab8a095ad653da47628" candidate_uuid: "c32b27909a9949f48f3ea22489973ace" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cff46e36bd9e400087ced7da2dbf9f0e"
I20260929 10:59:42.034662 2308 raft_consensus.cc:3037] T f1c7f48759624ab8a095ad653da47628 P 297c51b2364b4beb90b2d1f5282ac70a [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:42.034680 2448 raft_consensus.cc:3037] T f1c7f48759624ab8a095ad653da47628 P cff46e36bd9e400087ced7da2dbf9f0e [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:42.035290 2448 raft_consensus.cc:2454] T f1c7f48759624ab8a095ad653da47628 P cff46e36bd9e400087ced7da2dbf9f0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c32b27909a9949f48f3ea22489973ace in term 1.
I20260929 10:59:42.035313 2308 raft_consensus.cc:2454] T f1c7f48759624ab8a095ad653da47628 P 297c51b2364b4beb90b2d1f5282ac70a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c32b27909a9949f48f3ea22489973ace in term 1.
I20260929 10:59:42.035485 2519 leader_election.cc:302] T f1c7f48759624ab8a095ad653da47628 P c32b27909a9949f48f3ea22489973ace [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297c51b2364b4beb90b2d1f5282ac70a, c32b27909a9949f48f3ea22489973ace; no voters:
I20260929 10:59:42.035594 2668 raft_consensus.cc:2781] T f1c7f48759624ab8a095ad653da47628 P c32b27909a9949f48f3ea22489973ace [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:59:42.035655 2668 raft_consensus.cc:686] T f1c7f48759624ab8a095ad653da47628 P c32b27909a9949f48f3ea22489973ace [term 1 LEADER]: Becoming Leader. State: Replica: c32b27909a9949f48f3ea22489973ace, State: Running, Role: LEADER
I20260929 10:59:42.035702 2668 consensus_queue.cc:237] T f1c7f48759624ab8a095ad653da47628 P c32b27909a9949f48f3ea22489973ace [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } }
I20260929 10:59:42.036231 2150 catalog_manager.cc:5495] T f1c7f48759624ab8a095ad653da47628 P c32b27909a9949f48f3ea22489973ace reported cstate change: term changed from 0 to 1, leader changed from <none> to c32b27909a9949f48f3ea22489973ace (127.26.120.195). New cstate: current_term: 1 leader_uuid: "c32b27909a9949f48f3ea22489973ace" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c32b27909a9949f48f3ea22489973ace" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 35405 } health_report { overall_health: HEALTHY } } }
I20260929 10:59:42.072345 2661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:42.069988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42344 (local address 127.26.120.195:35405)
0929 10:59:42.070034 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:42.070036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:42.070080 (+ 44us) server_negotiation.cc:407] Connection header received
0929 10:59:42.070180 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:42.070181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:42.070191 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:42.070266 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:59:42.070390 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:42.070868 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:42.071614 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:42.071813 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:42.071893 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:42.072068 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:59:42.072109 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:59:42.072174 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:42.072235 (+ 61us) server_negotiation.cc:299] Negotiation successful
0929 10:59:42.072257 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 10:59:42.076624 2661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:42.074743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:58121 (local address 127.26.120.195:35405)
0929 10:59:42.074811 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:42.074813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:42.074820 (+ 7us) server_negotiation.cc:407] Connection header received
0929 10:59:42.074837 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:42.074837 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:42.074844 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:42.074887 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 10:59:42.074961 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:42.075398 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:42.076073 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:42.076241 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:42.076291 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:42.076417 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 10:59:42.076440 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 10:59:42.076477 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:42.076538 (+ 61us) server_negotiation.cc:299] Negotiation successful
0929 10:59:42.076555 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 10:59:42.076627 2640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:42.074639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:35405 (local address 127.26.120.195:58121)
0929 10:59:42.074695 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:42.074706 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:42.074752 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:42.074890 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:42.074891 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:42.074891 (+ 0us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 10:59:42.074925 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:42.074926 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:42.075464 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:42.075464 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:42.075988 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:42.075989 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:42.076307 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:42.076308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:42.076347 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:42.076351 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 10:59:42.076501 (+ 150us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 10:59:42.076502 (+ 1us) client_negotiation.cc:769] Sending connection context
0929 10:59:42.076535 (+ 33us) client_negotiation.cc:240] Negotiation successful
0929 10:59:42.076566 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260929 10:59:42.078567 2448 raft_consensus.cc:1261] T f1c7f48759624ab8a095ad653da47628 P cff46e36bd9e400087ced7da2dbf9f0e [term 1 FOLLOWER]: Refusing update from remote peer c32b27909a9949f48f3ea22489973ace: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:59:42.078604 2308 raft_consensus.cc:1261] T f1c7f48759624ab8a095ad653da47628 P 297c51b2364b4beb90b2d1f5282ac70a [term 1 FOLLOWER]: Refusing update from remote peer c32b27909a9949f48f3ea22489973ace: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 10:59:42.078787 2668 consensus_queue.cc:1035] T f1c7f48759624ab8a095ad653da47628 P c32b27909a9949f48f3ea22489973ace [LEADER]: Connected to new peer: Peer: permanent_uuid: "cff46e36bd9e400087ced7da2dbf9f0e" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 45525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:59:42.078877 2702 consensus_queue.cc:1035] T f1c7f48759624ab8a095ad653da47628 P c32b27909a9949f48f3ea22489973ace [LEADER]: Connected to new peer: Peer: permanent_uuid: "297c51b2364b4beb90b2d1f5282ac70a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 10:59:42.092186 2151 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:59:42.092288 2151 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 10:59:42.092931 2151 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5b33e54e771e4830afa648431a0880a2: Sending DeleteTablet for 3 replicas of tablet f1c7f48759624ab8a095ad653da47628
I20260929 10:59:42.093214 2288 tablet_service.cc:1514] Processing DeleteTablet for tablet f1c7f48759624ab8a095ad653da47628 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:42 UTC) from {username='oryx', principal='oryx/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:33744
I20260929 10:59:42.093220 2428 tablet_service.cc:1514] Processing DeleteTablet for tablet f1c7f48759624ab8a095ad653da47628 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:42 UTC) from {username='oryx', principal='oryx/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:41072
I20260929 10:59:42.093321 2687 tablet_replica.cc:330] T f1c7f48759624ab8a095ad653da47628 P 297c51b2364b4beb90b2d1f5282ac70a: stopping tablet replica
I20260929 10:59:42.093374 2687 raft_consensus.cc:2229] T f1c7f48759624ab8a095ad653da47628 P 297c51b2364b4beb90b2d1f5282ac70a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:59:42.093420 2687 raft_consensus.cc:2258] T f1c7f48759624ab8a095ad653da47628 P 297c51b2364b4beb90b2d1f5282ac70a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:42.093648 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 2223
I20260929 10:59:42.097318 2686 tablet_replica.cc:330] T f1c7f48759624ab8a095ad653da47628 P cff46e36bd9e400087ced7da2dbf9f0e: stopping tablet replica
I20260929 10:59:42.097411 2686 raft_consensus.cc:2229] T f1c7f48759624ab8a095ad653da47628 P cff46e36bd9e400087ced7da2dbf9f0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 10:59:42.097458 2686 raft_consensus.cc:2258] T f1c7f48759624ab8a095ad653da47628 P cff46e36bd9e400087ced7da2dbf9f0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:42.097777 2686 ts_tablet_manager.cc:1905] T f1c7f48759624ab8a095ad653da47628 P cff46e36bd9e400087ced7da2dbf9f0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:42.097801 2568 tablet_service.cc:1514] Processing DeleteTablet for tablet f1c7f48759624ab8a095ad653da47628 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 10:59:42 UTC) from {username='oryx', principal='oryx/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:42314
I20260929 10:59:42.097877 2685 tablet_replica.cc:330] T f1c7f48759624ab8a095ad653da47628 P c32b27909a9949f48f3ea22489973ace: stopping tablet replica
I20260929 10:59:42.097925 2685 raft_consensus.cc:2229] T f1c7f48759624ab8a095ad653da47628 P c32b27909a9949f48f3ea22489973ace [term 1 LEADER]: Raft consensus shutting down.
I20260929 10:59:42.098027 2685 raft_consensus.cc:2258] T f1c7f48759624ab8a095ad653da47628 P c32b27909a9949f48f3ea22489973ace [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 10:59:42.099646 2686 ts_tablet_manager.cc:1918] T f1c7f48759624ab8a095ad653da47628 P cff46e36bd9e400087ced7da2dbf9f0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:59:42.099711 2686 log.cc:1196] T f1c7f48759624ab8a095ad653da47628 P cff46e36bd9e400087ced7da2dbf9f0e: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-1/wal/wals/f1c7f48759624ab8a095ad653da47628
I20260929 10:59:42.099921 2686 ts_tablet_manager.cc:1939] T f1c7f48759624ab8a095ad653da47628 P cff46e36bd9e400087ced7da2dbf9f0e: Deleting consensus metadata
I20260929 10:59:42.100272 2685 ts_tablet_manager.cc:1905] T f1c7f48759624ab8a095ad653da47628 P c32b27909a9949f48f3ea22489973ace: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 10:59:42.100469 2138 catalog_manager.cc:4841] TS cff46e36bd9e400087ced7da2dbf9f0e (127.26.120.194:45525): tablet f1c7f48759624ab8a095ad653da47628 (table test-table [id=16a0f197caf948f6882855dc53324aef]) successfully deleted
I20260929 10:59:42.101650 2685 ts_tablet_manager.cc:1918] T f1c7f48759624ab8a095ad653da47628 P c32b27909a9949f48f3ea22489973ace: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 10:59:42.101698 2685 log.cc:1196] T f1c7f48759624ab8a095ad653da47628 P c32b27909a9949f48f3ea22489973ace: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790679549986039-27107-0/minicluster-data/ts-2/wal/wals/f1c7f48759624ab8a095ad653da47628
I20260929 10:59:42.101907 2685 ts_tablet_manager.cc:1939] T f1c7f48759624ab8a095ad653da47628 P c32b27909a9949f48f3ea22489973ace: Deleting consensus metadata
I20260929 10:59:42.102324 2137 catalog_manager.cc:4841] TS c32b27909a9949f48f3ea22489973ace (127.26.120.195:35405): tablet f1c7f48759624ab8a095ad653da47628 (table test-table [id=16a0f197caf948f6882855dc53324aef]) successfully deleted
W20260929 10:59:42.103155 2135 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.120.193:46459 (error 108)
I20260929 10:59:42.103336 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 2363
I20260929 10:59:42.105748 2750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:42.105523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:46459 (local address 127.0.0.1:33766)
0929 10:59:42.105637 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:42.105681 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.193:46459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":52,"threads_started":1}
W20260929 10:59:42.109635 2135 catalog_manager.cc:4584] TS 297c51b2364b4beb90b2d1f5282ac70a (127.26.120.193:46459): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f1c7f48759624ab8a095ad653da47628: Network error: Client connection negotiation failed: client connection to 127.26.120.193:46459: connect: Connection refused (error 111)
I20260929 10:59:42.111554 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 2503
I20260929 10:59:42.116803 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 2119
2026-09-29T10:59:42Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2519 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:59:42 dist-test-slave-2lzq krb5kdc[2753](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:59:42 dist-test-slave-2lzq krb5kdc[2753](info): set up 2 sockets
Sep 29 10:59:42 dist-test-slave-2lzq krb5kdc[2753](info): commencing operation
krb5kdc: starting...
W20260929 10:59:43.150833 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 10:59:43 dist-test-slave-2lzq krb5kdc[2753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:59:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:59:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.120.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
I20260929 10:59:43.266676 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:38449
--webserver_interface=127.26.120.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:42815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:38449
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:43.347342 2766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:43.347545 2766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:43.347568 2766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:43.348729 2766 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:59:43.348773 2766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:43.348788 2766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:43.348840 2766 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:59:43.348869 2766 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:59:43.350356 2766 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:42815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:38449
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:38449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:43.350625 2766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:43.350879 2766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:43.355772 2772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:43.355789 2771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:43.355926 2766 server_base.cc:1034] running on GCE node
W20260929 10:59:43.355785 2774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:43.356223 2766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:43.356510 2766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:43.357704 2766 hybrid_clock.cc:648] HybridClock initialized: now 1790679583357681 us; error 37 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-29T10:59:43Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1246 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 10:59:43 dist-test-slave-2lzq krb5kdc[2779](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 10:59:43 dist-test-slave-2lzq krb5kdc[2779](info): set up 2 sockets
Sep 29 10:59:43 dist-test-slave-2lzq krb5kdc[2779](info): commencing operation
krb5kdc: starting...
W20260929 10:59:44.398936 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 10:59:44 dist-test-slave-2lzq krb5kdc[2779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679584, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T10:59:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T10:59:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.120.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
I20260929 10:59:44.521435 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:39675
--webserver_interface=127.26.120.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:33299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:39675
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:44.601935 2792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:44.602211 2792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:44.602238 2792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:44.603343 2792 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:59:44.603377 2792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:44.603392 2792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:44.603405 2792 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:59:44.603417 2792 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:59:44.604805 2792 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:33299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:39675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:39675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:44.604987 2792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:44.605140 2792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:44.609995 2798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:44.610004 2797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:44.610138 2792 server_base.cc:1034] running on GCE node
W20260929 10:59:44.610009 2800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:44.610539 2792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:44.610831 2792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:44.611994 2792 hybrid_clock.cc:648] HybridClock initialized: now 1790679584611983 us; error 43 us; skew 500 ppm
Sep 29 10:59:44 dist-test-slave-2lzq krb5kdc[2779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679584, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:44.614372 2792 init.cc:377] Logged in from keytab as kudu/127.26.120.254@KRBTEST.COM (short username kudu)
I20260929 10:59:44.614830 2792 webserver.cc:460] Webserver started at http://127.26.120.254:34293/ using document root <none> and password file <none>
I20260929 10:59:44.615023 2792 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:44.615064 2792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:44.615151 2792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:44.616120 2792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "1a2e41deb6054432bfe86aea1965f6c7"
format_stamp: "Formatted at 2026-09-29 10:59:44 on dist-test-slave-2lzq"
server_key: "1065a5dfb47562663f2d10e626fe0bb3"
server_key_iv: "50a7717f96263eb503b2206406144573"
server_key_version: "encryptionkey@0"
I20260929 10:59:44.616421 2792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "1a2e41deb6054432bfe86aea1965f6c7"
format_stamp: "Formatted at 2026-09-29 10:59:44 on dist-test-slave-2lzq"
server_key: "1065a5dfb47562663f2d10e626fe0bb3"
server_key_iv: "50a7717f96263eb503b2206406144573"
server_key_version: "encryptionkey@0"
I20260929 10:59:44.617938 2792 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 10:59:44.618736 2807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:44.618920 2792 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:59:44.619014 2792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "1a2e41deb6054432bfe86aea1965f6c7"
format_stamp: "Formatted at 2026-09-29 10:59:44 on dist-test-slave-2lzq"
server_key: "1065a5dfb47562663f2d10e626fe0bb3"
server_key_iv: "50a7717f96263eb503b2206406144573"
server_key_version: "encryptionkey@0"
I20260929 10:59:44.619069 2792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:44.638788 2792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:44.641078 2792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:44.641187 2792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:44.645041 2792 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:39675
I20260929 10:59:44.645098 2859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:39675 every 8 connection(s)
I20260929 10:59:44.645500 2792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 10:59:44.646293 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 2792
I20260929 10:59:44.646384 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 10:59:44.646499 2860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 10:59:44.646624 27107 external_mini_cluster.cc:1428] Setting key 3a4f8ff59e5f484c15073acc0cd42199
I20260929 10:59:44.649636 2860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7: Bootstrap starting.
I20260929 10:59:44.650406 2860 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7: Neither blocks nor log segments found. Creating new log.
I20260929 10:59:44.650722 2860 log.cc:824] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:44.651468 2860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7: No bootstrap required, opened a new log
I20260929 10:59:44.653429 2860 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a2e41deb6054432bfe86aea1965f6c7" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 39675 } }
I20260929 10:59:44.653599 2860 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
Sep 29 10:59:44 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679584, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:44.653630 2860 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a2e41deb6054432bfe86aea1965f6c7, State: Initialized, Role: FOLLOWER
I20260929 10:59:44.653753 2860 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a2e41deb6054432bfe86aea1965f6c7" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 39675 } }
I20260929 10:59:44.653868 2860 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:59:44.653913 2860 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:59:44.653961 2860 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 10:59:44.654557 2860 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a2e41deb6054432bfe86aea1965f6c7" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 39675 } }
I20260929 10:59:44.654678 2860 leader_election.cc:302] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a2e41deb6054432bfe86aea1965f6c7; no voters:
I20260929 10:59:44.654886 2860 leader_election.cc:288] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 10:59:44.654915 2865 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 10:59:44.655020 2865 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7 [term 1 LEADER]: Becoming Leader. State: Replica: 1a2e41deb6054432bfe86aea1965f6c7, State: Running, Role: LEADER
I20260929 10:59:44.655105 2865 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a2e41deb6054432bfe86aea1965f6c7" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 39675 } }
I20260929 10:59:44.655151 2860 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:59:44.655405 2865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a2e41deb6054432bfe86aea1965f6c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a2e41deb6054432bfe86aea1965f6c7" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 39675 } } }
I20260929 10:59:44.655478 2865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7 [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:44.655586 2865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a2e41deb6054432bfe86aea1965f6c7. Latest consensus state: current_term: 1 leader_uuid: "1a2e41deb6054432bfe86aea1965f6c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a2e41deb6054432bfe86aea1965f6c7" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 39675 } } }
I20260929 10:59:44.655685 2865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7 [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:44.655835 2870 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:59:44.656361 2870 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:59:44.658035 2870 catalog_manager.cc:1347] Generated new cluster ID: 823387fdf3bb433cab422a87a9075c48
I20260929 10:59:44.658114 2870 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:59:44.658278 2863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:44.647269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43240 (local address 127.26.120.254:39675)
0929 10:59:44.647453 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:44.647470 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:44.647496 (+ 26us) server_negotiation.cc:407] Connection header received
0929 10:59:44.647974 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:44.647981 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:44.648025 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:44.648129 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:44.648943 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:44.649858 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:44.651001 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:44.651499 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:44.651689 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:44.654202 (+ 2513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:44.654207 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:44.654208 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:44.654223 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:44.656191 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:44.656560 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:44.656563 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:44.656563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:44.656750 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:44.656970 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:44.656971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:44.656971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:44.657257 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:44.657816 (+ 559us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:44.657882 (+ 66us) server_negotiation.cc:299] Negotiation successful
0929 10:59:44.658037 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":91,"threads_started":1}
I20260929 10:59:44.699347 2870 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 10:59:44.700317 2870 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:59:44.714161 2870 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7: Generated new TSK 0
I20260929 10:59:44.714394 2870 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.120.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab.
Entry for principal kudu/127.26.120.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab.
I20260929 10:59:44.749233 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.193:0
--local_ip_for_outbound_sockets=127.26.120.193
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:39675
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:33299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:44.850317 2886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:44.850513 2886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:44.850536 2886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:44.851651 2886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:44.851686 2886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:44.851712 2886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.193
I20260929 10:59:44.853029 2886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:33299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:39675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.193
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:44.853201 2886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:44.853377 2886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:44.858568 2891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:44.858731 2886 server_base.cc:1034] running on GCE node
W20260929 10:59:44.858593 2892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:44.858559 2894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:44.859017 2886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:44.859266 2886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:44.860440 2886 hybrid_clock.cc:648] HybridClock initialized: now 1790679584860431 us; error 30 us; skew 500 ppm
Sep 29 10:59:44 dist-test-slave-2lzq krb5kdc[2779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679584, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:44.863009 2886 init.cc:377] Logged in from keytab as kudu/127.26.120.193@KRBTEST.COM (short username kudu)
I20260929 10:59:44.863461 2886 webserver.cc:460] Webserver started at http://127.26.120.193:39303/ using document root <none> and password file <none>
I20260929 10:59:44.863688 2886 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:44.863735 2886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:44.863901 2886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:44.865341 2886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-0/data/instance:
uuid: "833753af11c545e4a4dc0b37b4680b57"
format_stamp: "Formatted at 2026-09-29 10:59:44 on dist-test-slave-2lzq"
server_key: "c07acae7523bb24f8fc31cd47794e380"
server_key_iv: "476861d8122ac21271e64d0d07f29c9e"
server_key_version: "encryptionkey@0"
I20260929 10:59:44.865751 2886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance:
uuid: "833753af11c545e4a4dc0b37b4680b57"
format_stamp: "Formatted at 2026-09-29 10:59:44 on dist-test-slave-2lzq"
server_key: "c07acae7523bb24f8fc31cd47794e380"
server_key_iv: "476861d8122ac21271e64d0d07f29c9e"
server_key_version: "encryptionkey@0"
I20260929 10:59:44.867141 2886 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:59:44.867923 2901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:44.868112 2886 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:59:44.868182 2886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-0/wal
uuid: "833753af11c545e4a4dc0b37b4680b57"
format_stamp: "Formatted at 2026-09-29 10:59:44 on dist-test-slave-2lzq"
server_key: "c07acae7523bb24f8fc31cd47794e380"
server_key_iv: "476861d8122ac21271e64d0d07f29c9e"
server_key_version: "encryptionkey@0"
I20260929 10:59:44.868271 2886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:44.885290 2886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:44.887682 2886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:44.887849 2886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:44.888093 2886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:59:44.888415 2886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:44.888471 2886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:44.888512 2886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:44.888540 2886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:44.894618 2886 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.193:34067
I20260929 10:59:44.894663 3014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.193:34067 every 8 connection(s)
I20260929 10:59:44.895025 2886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
Sep 29 10:59:44 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679584, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:44.904894 2863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:44.895725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:58529 (local address 127.26.120.254:39675)
0929 10:59:44.895765 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:44.895767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:44.895973 (+ 206us) server_negotiation.cc:407] Connection header received
0929 10:59:44.896708 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:44.896709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:44.896723 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:44.896770 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:44.897964 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:44.898593 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:44.900014 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:44.900243 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:44.900320 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:44.902378 (+ 2058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:44.902378 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:44.902379 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:44.902385 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:44.903791 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:44.904140 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:44.904141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:44.904141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:44.904191 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:44.904393 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:44.904393 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:44.904393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:44.904490 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:44.904540 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:44.904743 (+ 203us) server_negotiation.cc:299] Negotiation successful
0929 10:59:44.904803 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260929 10:59:44.905082 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 2886
I20260929 10:59:44.905113 3017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:44.895699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.193:58529)
0929 10:59:44.895838 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:44.895885 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:44.896563 (+ 678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:44.896839 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:44.896843 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:44.897650 (+ 807us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:44.897844 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:44.897849 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:44.898678 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:44.898679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:44.899927 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:44.899929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:44.900340 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:44.900340 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:44.900476 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:44.900905 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:44.900913 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:44.902283 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:44.903861 (+ 1578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:44.903867 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:44.903868 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:44.904072 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:44.904248 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:44.904248 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:44.904249 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:44.904338 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:44.904543 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:44.904548 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:44.904653 (+ 105us) client_negotiation.cc:769] Sending connection context
0929 10:59:44.904731 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 10:59:44.904841 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":71,"threads_started":1}
I20260929 10:59:44.905157 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance
I20260929 10:59:44.905359 27107 external_mini_cluster.cc:1428] Setting key ea50e0cd78119865a5e936fe5dbec9aa
I20260929 10:59:44.905750 3015 heartbeater.cc:347] Connected to a master server at 127.26.120.254:39675
I20260929 10:59:44.905850 3015 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:44.906075 3015 heartbeater.cc:511] Master 127.26.120.254:39675 requested a full tablet report, sending...
I20260929 10:59:44.906656 2824 ts_manager.cc:194] Registered new tserver with Master: 833753af11c545e4a4dc0b37b4680b57 (127.26.120.193:34067)
I20260929 10:59:44.907274 2824 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.193@KRBTEST.COM'} at 127.26.120.193:58529
WARNING: no policy specified for kudu/127.26.120.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab.
Entry for principal kudu/127.26.120.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab.
I20260929 10:59:44.931025 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.194:0
--local_ip_for_outbound_sockets=127.26.120.194
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:39675
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:33299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:45.010172 3020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:45.010413 3020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:45.010450 3020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:45.011639 3020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:45.011715 3020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:45.011760 3020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.194
I20260929 10:59:45.013198 3020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:33299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:39675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.194
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:45.013476 3020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:45.013718 3020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:45.018714 3025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:45.018805 3020 server_base.cc:1034] running on GCE node
W20260929 10:59:45.018725 3028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:45.018754 3026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:45.019158 3020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:45.019464 3020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:45.020643 3020 hybrid_clock.cc:648] HybridClock initialized: now 1790679585020628 us; error 28 us; skew 500 ppm
Sep 29 10:59:45 dist-test-slave-2lzq krb5kdc[2779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679585, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:45.023203 3020 init.cc:377] Logged in from keytab as kudu/127.26.120.194@KRBTEST.COM (short username kudu)
I20260929 10:59:45.023650 3020 webserver.cc:460] Webserver started at http://127.26.120.194:41349/ using document root <none> and password file <none>
I20260929 10:59:45.023895 3020 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:45.023964 3020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:45.024099 3020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:45.025146 3020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-1/data/instance:
uuid: "bbe75f11d01149e5820ded2eb1407b63"
format_stamp: "Formatted at 2026-09-29 10:59:45 on dist-test-slave-2lzq"
server_key: "43f88f9313006bfef6e1995689b3b5e5"
server_key_iv: "173ce10713252fd1f116ca44c869924d"
server_key_version: "encryptionkey@0"
I20260929 10:59:45.025534 3020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance:
uuid: "bbe75f11d01149e5820ded2eb1407b63"
format_stamp: "Formatted at 2026-09-29 10:59:45 on dist-test-slave-2lzq"
server_key: "43f88f9313006bfef6e1995689b3b5e5"
server_key_iv: "173ce10713252fd1f116ca44c869924d"
server_key_version: "encryptionkey@0"
I20260929 10:59:45.026964 3020 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 10:59:45.027734 3035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:45.027907 3020 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 10:59:45.028005 3020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-1/wal
uuid: "bbe75f11d01149e5820ded2eb1407b63"
format_stamp: "Formatted at 2026-09-29 10:59:45 on dist-test-slave-2lzq"
server_key: "43f88f9313006bfef6e1995689b3b5e5"
server_key_iv: "173ce10713252fd1f116ca44c869924d"
server_key_version: "encryptionkey@0"
I20260929 10:59:45.028085 3020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:45.038228 3020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:45.040112 3020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:45.040278 3020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:45.040508 3020 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:59:45.040817 3020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:45.040869 3020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:45.040905 3020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:45.040930 3020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:45.046970 3020 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.194:46527
I20260929 10:59:45.047053 3148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.194:46527 every 8 connection(s)
I20260929 10:59:45.047376 3020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
Sep 29 10:59:45 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679585, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:45.055833 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 3020
I20260929 10:59:45.055956 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance
I20260929 10:59:45.056129 27107 external_mini_cluster.cc:1428] Setting key 69d2a5b9392a41d4dccbb37ca3999fcf
I20260929 10:59:45.057289 2863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:45.048123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:50399 (local address 127.26.120.254:39675)
0929 10:59:45.048227 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:45.048230 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:45.048398 (+ 168us) server_negotiation.cc:407] Connection header received
0929 10:59:45.049050 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:45.049051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:45.049065 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:45.049116 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:45.050090 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.050627 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.051657 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.051871 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.051962 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:45.054204 (+ 2242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:45.054206 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:45.054206 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:45.054214 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:45.055738 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:45.056281 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:45.056282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:45.056283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:45.056346 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:45.056616 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:45.056617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:45.056617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:45.056740 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:45.056823 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:45.057110 (+ 287us) server_negotiation.cc:299] Negotiation successful
0929 10:59:45.057199 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 10:59:45.057526 3151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:45.048037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.194:50399)
0929 10:59:45.048258 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:45.048306 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:45.048929 (+ 623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:45.049180 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:45.049184 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:45.049746 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:45.049965 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:45.049970 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.050731 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.050732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:45.051546 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:45.051548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.051970 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.051971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:45.052101 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:45.052652 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:45.052660 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:45.054035 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:45.055829 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:45.055835 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:45.055838 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:45.056144 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:45.056417 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:45.056419 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:45.056419 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:45.056540 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:45.056813 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:45.056819 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:45.056962 (+ 143us) client_negotiation.cc:769] Sending connection context
0929 10:59:45.057088 (+ 126us) client_negotiation.cc:240] Negotiation successful
0929 10:59:45.057248 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":117,"threads_started":1}
I20260929 10:59:45.058566 3149 heartbeater.cc:347] Connected to a master server at 127.26.120.254:39675
I20260929 10:59:45.058684 3149 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:45.058863 3149 heartbeater.cc:511] Master 127.26.120.254:39675 requested a full tablet report, sending...
I20260929 10:59:45.059257 2824 ts_manager.cc:194] Registered new tserver with Master: bbe75f11d01149e5820ded2eb1407b63 (127.26.120.194:46527)
I20260929 10:59:45.059641 2824 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.194@KRBTEST.COM'} at 127.26.120.194:50399
WARNING: no policy specified for kudu/127.26.120.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab.
Entry for principal kudu/127.26.120.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab.
I20260929 10:59:45.081677 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.195:0
--local_ip_for_outbound_sockets=127.26.120.195
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:39675
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:33299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:45.164103 3154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:45.164305 3154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:45.164330 3154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:45.165506 3154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:45.165551 3154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:45.165577 3154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.195
I20260929 10:59:45.167059 3154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:33299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:39675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.195
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:45.167306 3154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:45.167541 3154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:45.172618 3160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:45.172742 3154 server_base.cc:1034] running on GCE node
W20260929 10:59:45.172609 3162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:45.172608 3159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:45.173250 3154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:45.173516 3154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:45.174656 3154 hybrid_clock.cc:648] HybridClock initialized: now 1790679585174644 us; error 28 us; skew 500 ppm
Sep 29 10:59:45 dist-test-slave-2lzq krb5kdc[2779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679585, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:45.177145 3154 init.cc:377] Logged in from keytab as kudu/127.26.120.195@KRBTEST.COM (short username kudu)
I20260929 10:59:45.177672 3154 webserver.cc:460] Webserver started at http://127.26.120.195:33405/ using document root <none> and password file <none>
I20260929 10:59:45.177906 3154 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:45.177978 3154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:45.178180 3154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 10:59:45.179255 3154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-2/data/instance:
uuid: "988bd6da9787461595fd445cb9cd21b5"
format_stamp: "Formatted at 2026-09-29 10:59:45 on dist-test-slave-2lzq"
server_key: "3542d4bc56b4157e10f343b0ecbec865"
server_key_iv: "fb38da7687886a463f7ea19102cbd3a5"
server_key_version: "encryptionkey@0"
I20260929 10:59:45.179628 3154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance:
uuid: "988bd6da9787461595fd445cb9cd21b5"
format_stamp: "Formatted at 2026-09-29 10:59:45 on dist-test-slave-2lzq"
server_key: "3542d4bc56b4157e10f343b0ecbec865"
server_key_iv: "fb38da7687886a463f7ea19102cbd3a5"
server_key_version: "encryptionkey@0"
I20260929 10:59:45.180946 3154 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 10:59:45.181663 3169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:45.181833 3154 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 10:59:45.181933 3154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-2/wal
uuid: "988bd6da9787461595fd445cb9cd21b5"
format_stamp: "Formatted at 2026-09-29 10:59:45 on dist-test-slave-2lzq"
server_key: "3542d4bc56b4157e10f343b0ecbec865"
server_key_iv: "fb38da7687886a463f7ea19102cbd3a5"
server_key_version: "encryptionkey@0"
I20260929 10:59:45.182019 3154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:45.192003 3154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:45.193950 3154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:45.194168 3154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:45.194386 3154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:59:45.194696 3154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:45.194753 3154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:45.194795 3154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:45.194823 3154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:45.200847 3154 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.195:34353
I20260929 10:59:45.200922 3282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.195:34353 every 8 connection(s)
I20260929 10:59:45.201231 3154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
I20260929 10:59:45.206813 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 3154
I20260929 10:59:45.206918 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance
I20260929 10:59:45.207093 27107 external_mini_cluster.cc:1428] Setting key 1f68fe967c9e3f543ad9699ac694e24f
Sep 29 10:59:45 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679585, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 10:59:45.210628 2863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:45.201962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:58411 (local address 127.26.120.254:39675)
0929 10:59:45.202034 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:45.202036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:45.202275 (+ 239us) server_negotiation.cc:407] Connection header received
0929 10:59:45.202748 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:45.202749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:45.202758 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:45.202823 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:45.203744 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.204231 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.205210 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.205385 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.205435 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:45.207803 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:45.207804 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:45.207805 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:45.207813 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:45.209121 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:45.209580 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:45.209582 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:45.209582 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:45.209645 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:45.209937 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:45.209938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:45.209939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:45.210109 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:45.210172 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:45.210455 (+ 283us) server_negotiation.cc:299] Negotiation successful
0929 10:59:45.210535 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 10:59:45.210829 3285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:45.201940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.195:58411)
0929 10:59:45.202143 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:45.202186 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:45.202667 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:45.202880 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:45.202883 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:45.203475 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:45.203667 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:45.203671 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.204312 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.204312 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:45.205135 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:45.205137 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.205472 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.205472 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:45.205567 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:45.206000 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:45.206008 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:45.207700 (+ 1692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 10:59:45.209240 (+ 1540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:45.209247 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:45.209249 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:45.209483 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:45.209717 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 10:59:45.209718 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 10:59:45.209719 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 10:59:45.209837 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 10:59:45.210164 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 10:59:45.210170 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 10:59:45.210313 (+ 143us) client_negotiation.cc:769] Sending connection context
0929 10:59:45.210431 (+ 118us) client_negotiation.cc:240] Negotiation successful
0929 10:59:45.210595 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":139,"threads_started":1}
I20260929 10:59:45.211263 3283 heartbeater.cc:347] Connected to a master server at 127.26.120.254:39675
I20260929 10:59:45.211341 3283 heartbeater.cc:464] Registering TS with master...
I20260929 10:59:45.211593 3283 heartbeater.cc:511] Master 127.26.120.254:39675 requested a full tablet report, sending...
I20260929 10:59:45.211994 2824 ts_manager.cc:194] Registered new tserver with Master: 988bd6da9787461595fd445cb9cd21b5 (127.26.120.195:34353)
I20260929 10:59:45.212384 2824 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.195@KRBTEST.COM'} at 127.26.120.195:58411
I20260929 10:59:45.218708 27107 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.26.120.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.120.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.120.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.193.keytab.
Entry for principal oryx/127.26.120.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.193.keytab.
WARNING: no policy specified for oryx/127.26.120.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.120.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.120.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.194.keytab.
Entry for principal oryx/127.26.120.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.194.keytab.
WARNING: no policy specified for oryx/127.26.120.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.120.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.120.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.195.keytab.
Entry for principal oryx/127.26.120.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.195.keytab.
I20260929 10:59:45.292099 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 2886
I20260929 10:59:45.296593 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 3020
I20260929 10:59:45.300922 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 3154
I20260929 10:59:45.304906 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 2792
I20260929 10:59:45.309746 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:39675
--webserver_interface=127.26.120.254
--webserver_port=34293
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:33299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:39675
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 10:59:45.390638 3292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:45.390838 3292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:45.390861 3292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:45.391932 3292 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 10:59:45.391968 3292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:45.391983 3292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:45.391996 3292 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 10:59:45.392009 3292 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 10:59:45.393394 3292 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:33299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:39675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:39675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=34293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:45.393576 3292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:45.393746 3292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:45.398684 3298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:45.398691 3297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:45.398691 3300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:45.398768 3292 server_base.cc:1034] running on GCE node
I20260929 10:59:45.399132 3292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:45.399472 3292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:45.400658 3292 hybrid_clock.cc:648] HybridClock initialized: now 1790679585400637 us; error 39 us; skew 500 ppm
Sep 29 10:59:45 dist-test-slave-2lzq krb5kdc[2779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679585, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:45.403292 3292 init.cc:377] Logged in from keytab as kudu/127.26.120.254@KRBTEST.COM (short username kudu)
I20260929 10:59:45.403791 3292 webserver.cc:460] Webserver started at http://127.26.120.254:34293/ using document root <none> and password file <none>
I20260929 10:59:45.404053 3292 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:45.404141 3292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:45.405530 3292 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:59:45.406288 3307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:45.406491 3292 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 10:59:45.406597 3292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "1a2e41deb6054432bfe86aea1965f6c7"
format_stamp: "Formatted at 2026-09-29 10:59:44 on dist-test-slave-2lzq"
server_key: "1065a5dfb47562663f2d10e626fe0bb3"
server_key_iv: "50a7717f96263eb503b2206406144573"
server_key_version: "encryptionkey@0"
I20260929 10:59:45.406976 3292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:45.419178 3292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:45.421139 3292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:45.421308 3292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:45.425621 3292 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:39675
I20260929 10:59:45.425660 3359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:39675 every 8 connection(s)
I20260929 10:59:45.426149 3292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 10:59:45.427615 3360 sys_catalog.cc:263] Verifying existing consensus state
I20260929 10:59:45.428217 3360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7: Bootstrap starting.
I20260929 10:59:45.434590 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 3292
I20260929 10:59:45.434949 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.193:34067
--local_ip_for_outbound_sockets=127.26.120.193
--tserver_master_addrs=127.26.120.254:39675
--webserver_port=39303
--webserver_interface=127.26.120.193
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:33299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.26.120.193
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 10:59:45.436719 3360 log.cc:824] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7: Log is configured to *not* fsync() on all Append() calls
I20260929 10:59:45.438791 3360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 10:59:45.439085 3360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7: Bootstrap complete.
I20260929 10:59:45.441322 3360 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a2e41deb6054432bfe86aea1965f6c7" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 39675 } }
I20260929 10:59:45.441538 3360 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a2e41deb6054432bfe86aea1965f6c7, State: Initialized, Role: FOLLOWER
I20260929 10:59:45.441707 3360 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a2e41deb6054432bfe86aea1965f6c7" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 39675 } }
I20260929 10:59:45.441785 3360 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 10:59:45.441850 3360 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 10:59:45.441936 3360 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7 [term 1 FOLLOWER]: Advancing to term 2
I20260929 10:59:45.442960 3360 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a2e41deb6054432bfe86aea1965f6c7" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 39675 } }
I20260929 10:59:45.443043 3360 leader_election.cc:302] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a2e41deb6054432bfe86aea1965f6c7; no voters:
I20260929 10:59:45.443251 3360 leader_election.cc:288] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 10:59:45.443287 3364 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 10:59:45.443413 3364 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7 [term 2 LEADER]: Becoming Leader. State: Replica: 1a2e41deb6054432bfe86aea1965f6c7, State: Running, Role: LEADER
I20260929 10:59:45.443522 3360 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 10:59:45.443567 3364 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a2e41deb6054432bfe86aea1965f6c7" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 39675 } }
I20260929 10:59:45.443878 3364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a2e41deb6054432bfe86aea1965f6c7. Latest consensus state: current_term: 2 leader_uuid: "1a2e41deb6054432bfe86aea1965f6c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a2e41deb6054432bfe86aea1965f6c7" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 39675 } } }
I20260929 10:59:45.444036 3364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7 [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:45.443883 3365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1a2e41deb6054432bfe86aea1965f6c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a2e41deb6054432bfe86aea1965f6c7" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 39675 } } }
I20260929 10:59:45.444197 3365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7 [sys.catalog]: This master's current role is: LEADER
I20260929 10:59:45.444579 3373 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 10:59:45.445163 3373 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 10:59:45.445348 3373 catalog_manager.cc:1259] Loaded cluster ID: 823387fdf3bb433cab422a87a9075c48
I20260929 10:59:45.445603 3373 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 10:59:45.446967 3373 catalog_manager.cc:1504] Loading token signing keys...
I20260929 10:59:45.447234 3373 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1a2e41deb6054432bfe86aea1965f6c7: Loaded TSK: 0
I20260929 10:59:45.447489 3373 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 10:59:45.523319 3362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:45.523664 3362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:45.523710 3362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:45.525317 3362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:45.525409 3362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:45.525460 3362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.193
I20260929 10:59:45.527100 3362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:33299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.193.keytab
--principal=oryx/127.26.120.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.193:34067
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.120.193
--webserver_port=39303
--tserver_master_addrs=127.26.120.254:39675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.193
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:45.527390 3362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:45.527631 3362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:45.532941 3387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:45.533097 3362 server_base.cc:1034] running on GCE node
W20260929 10:59:45.532918 3386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:45.532944 3389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:45.533507 3362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:45.533766 3362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:45.534955 3362 hybrid_clock.cc:648] HybridClock initialized: now 1790679585534940 us; error 30 us; skew 500 ppm
Sep 29 10:59:45 dist-test-slave-2lzq krb5kdc[2779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679585, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:45.537665 3362 init.cc:377] Logged in from keytab as oryx/127.26.120.193@KRBTEST.COM (short username oryx)
I20260929 10:59:45.538198 3362 webserver.cc:460] Webserver started at http://127.26.120.193:39303/ using document root <none> and password file <none>
I20260929 10:59:45.538448 3362 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:45.538543 3362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:45.540072 3362 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 10:59:45.540766 3396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:45.540985 3362 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:59:45.541090 3362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-0/wal
uuid: "833753af11c545e4a4dc0b37b4680b57"
format_stamp: "Formatted at 2026-09-29 10:59:44 on dist-test-slave-2lzq"
server_key: "c07acae7523bb24f8fc31cd47794e380"
server_key_iv: "476861d8122ac21271e64d0d07f29c9e"
server_key_version: "encryptionkey@0"
I20260929 10:59:45.541486 3362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:45.560860 3362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:45.563235 3362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:45.563431 3362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:45.563697 3362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:59:45.564025 3362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:45.564085 3362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:45.564128 3362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:45.564150 3362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:45.570847 3362 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.193:34067
I20260929 10:59:45.570905 3509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.193:34067 every 8 connection(s)
I20260929 10:59:45.571296 3362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
I20260929 10:59:45.572898 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 3362
I20260929 10:59:45.573220 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.194:46527
--local_ip_for_outbound_sockets=127.26.120.194
--tserver_master_addrs=127.26.120.254:39675
--webserver_port=41349
--webserver_interface=127.26.120.194
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:33299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.26.120.194
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 10:59:45 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:45 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:45.582299 3512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:45.572128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.193:36743)
0929 10:59:45.572358 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:45.572410 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:45.573052 (+ 642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:45.573597 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:45.573601 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:45.574352 (+ 751us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:45.574618 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:45.574623 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.576242 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.576244 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:45.577256 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:45.577259 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.578289 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.578290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:45.578465 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:45.579087 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:45.579095 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:45.582094 (+ 2999us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":114,"threads_started":1}
W20260929 10:59:45.582759 3510 heartbeater.cc:644] Failed to heartbeat to 127.26.120.254:39675 (0 consecutive failures): Not authorized: Failed to ping master at 127.26.120.254:39675: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
W20260929 10:59:45.583209 3513 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:45.572279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:36743 (local address 127.26.120.254:39675)
0929 10:59:45.572445 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:45.572454 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:45.572491 (+ 37us) server_negotiation.cc:407] Connection header received
0929 10:59:45.573290 (+ 799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:45.573299 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:45.573369 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:45.573517 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:45.574749 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.576118 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.577378 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.578035 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.578169 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:45.582278 (+ 4109us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.193:36743: BlockingRecv error: recv got EOF from 127.26.120.193:36743 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":79,"threads_started":1}
Sep 29 10:59:45 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:45 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:45.588486 3512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:45.583030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.193:36367)
0929 10:59:45.583232 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:45.583244 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:45.583293 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:45.583882 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:45.583883 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:45.584044 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:45.584106 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:45.584108 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.585141 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.585142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:45.585656 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:45.585658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.586182 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.586183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:45.586240 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:45.586739 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:45.586744 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:45.588439 (+ 1695us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183}
W20260929 10:59:45.588812 3513 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:45.583088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:36367 (local address 127.26.120.254:39675)
0929 10:59:45.583728 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:45.583730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:45.583741 (+ 11us) server_negotiation.cc:407] Connection header received
0929 10:59:45.583769 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:45.583770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:45.583785 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:45.583852 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:45.584218 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.585052 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.585742 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.586105 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.586182 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:45.588549 (+ 2367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.193:36367: BlockingRecv error: recv got EOF from 127.26.120.193:36367 (error 108)
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":106,"threads_started":1}
Sep 29 10:59:45 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:45 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:45.593277 3512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:45.588741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.193:46077)
0929 10:59:45.588796 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:45.588808 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:45.588852 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:45.589025 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:45.589026 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:45.589129 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:45.589154 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:45.589156 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.590031 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.590032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:45.590559 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:45.590561 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.591016 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.591017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:45.591062 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:45.591484 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:45.591488 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:45.593195 (+ 1707us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260929 10:59:45.593312 3513 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:45.588819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:46077 (local address 127.26.120.254:39675)
0929 10:59:45.588909 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:45.588911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:45.588919 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:59:45.588939 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:45.588940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:45.588951 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:45.589006 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:45.589224 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.589954 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.590630 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.590927 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.590993 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:45.593272 (+ 2279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.193:46077: BlockingRecv error: recv got EOF from 127.26.120.193:46077 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260929 10:59:45.593441 3510 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 10:59:45.662487 3514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:45.662696 3514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:45.662719 3514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:45.663820 3514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:45.663854 3514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:45.663887 3514 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.194
I20260929 10:59:45.665223 3514 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:33299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.194.keytab
--principal=oryx/127.26.120.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.194:46527
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.120.194
--webserver_port=41349
--tserver_master_addrs=127.26.120.254:39675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3514
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.194
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:45.665403 3514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:45.665570 3514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:45.670444 3520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:45.670423 3521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:45.670413 3523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:45.670470 3514 server_base.cc:1034] running on GCE node
I20260929 10:59:45.670931 3514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:45.671185 3514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:45.672361 3514 hybrid_clock.cc:648] HybridClock initialized: now 1790679585672340 us; error 43 us; skew 500 ppm
Sep 29 10:59:45 dist-test-slave-2lzq krb5kdc[2779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679585, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:45.675007 3514 init.cc:377] Logged in from keytab as oryx/127.26.120.194@KRBTEST.COM (short username oryx)
I20260929 10:59:45.675390 3514 webserver.cc:460] Webserver started at http://127.26.120.194:41349/ using document root <none> and password file <none>
I20260929 10:59:45.675613 3514 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:45.675691 3514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:45.677027 3514 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:59:45.677687 3530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:45.677968 3514 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:59:45.678082 3514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-1/wal
uuid: "bbe75f11d01149e5820ded2eb1407b63"
format_stamp: "Formatted at 2026-09-29 10:59:45 on dist-test-slave-2lzq"
server_key: "43f88f9313006bfef6e1995689b3b5e5"
server_key_iv: "173ce10713252fd1f116ca44c869924d"
server_key_version: "encryptionkey@0"
I20260929 10:59:45.678455 3514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:45.692963 3514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:45.695020 3514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:45.695173 3514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:45.695397 3514 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:59:45.695714 3514 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:45.695767 3514 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:45.695809 3514 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:45.695835 3514 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:45.701567 3514 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.194:46527
I20260929 10:59:45.701642 3643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.194:46527 every 8 connection(s)
I20260929 10:59:45.701977 3514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
Sep 29 10:59:45 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:45 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
W20260929 10:59:45.709128 3513 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:45.702710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:52939 (local address 127.26.120.254:39675)
0929 10:59:45.702766 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:45.702768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:45.702999 (+ 231us) server_negotiation.cc:407] Connection header received
0929 10:59:45.703446 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:45.703447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:45.703460 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:45.703524 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:45.704397 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.704875 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.705812 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.705999 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.706090 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:45.709052 (+ 2962us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.194:52939: BlockingRecv error: recv got EOF from 127.26.120.194:52939 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
I20260929 10:59:45.709168 3646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:45.702664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.194:52939)
0929 10:59:45.702834 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:45.702900 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:45.703364 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:45.703575 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:45.703578 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:45.704134 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:45.704319 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:45.704324 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.704955 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.704956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:45.705737 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:45.705739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.706146 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.706147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:45.706266 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:45.706824 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:45.706833 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:45.708974 (+ 2141us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":96,"threads_started":1}
W20260929 10:59:45.709367 3644 heartbeater.cc:644] Failed to heartbeat to 127.26.120.254:39675 (0 consecutive failures): Not authorized: Failed to ping master at 127.26.120.254:39675: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
I20260929 10:59:45.711473 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 3514
I20260929 10:59:45.711755 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.195:34353
--local_ip_for_outbound_sockets=127.26.120.195
--tserver_master_addrs=127.26.120.254:39675
--webserver_port=33405
--webserver_interface=127.26.120.195
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:33299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.26.120.195
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 10:59:45 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:45 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:45.714164 3646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:45.709611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.194:39021)
0929 10:59:45.709668 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:45.709680 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:45.709727 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:45.709868 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:45.709869 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:45.709975 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:45.710017 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:45.710018 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.710740 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.710741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:45.711293 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:45.711295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.711742 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.711742 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:45.711789 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:45.712229 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:45.712235 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:45.714094 (+ 1859us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260929 10:59:45.714336 3513 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:45.709677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:39021 (local address 127.26.120.254:39675)
0929 10:59:45.709735 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:45.709736 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:45.709747 (+ 11us) server_negotiation.cc:407] Connection header received
0929 10:59:45.709778 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:45.709779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:45.709786 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:45.709859 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:45.710154 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.710673 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.711380 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.711673 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.711744 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:45.714275 (+ 2531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.194:39021: BlockingRecv error: recv got EOF from 127.26.120.194:39021 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 29 10:59:45 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:45 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:45.719655 3646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:45.714638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.194:51519)
0929 10:59:45.714672 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:45.714685 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:45.714758 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:45.714902 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:45.714902 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:45.715053 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:45.715092 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:45.715093 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.715757 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.715759 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:45.716677 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:45.716680 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.717067 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.717068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:45.717134 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:45.717531 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:45.717536 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:45.719559 (+ 2023us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":16}
W20260929 10:59:45.719693 3513 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:45.714684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:51519 (local address 127.26.120.254:39675)
0929 10:59:45.714721 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:45.714723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:45.714737 (+ 14us) server_negotiation.cc:407] Connection header received
0929 10:59:45.714811 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:45.714811 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:45.714819 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:45.714892 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:45.715160 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.715668 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.716778 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.716982 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.717047 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:45.719651 (+ 2604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.194:51519: BlockingRecv error: recv got EOF from 127.26.120.194:51519 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260929 10:59:45.719980 3644 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 10:59:45.793732 3647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 10:59:45.793927 3647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 10:59:45.793951 3647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 10:59:45.795213 3647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 10:59:45.795270 3647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 10:59:45.795313 3647 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.195
I20260929 10:59:45.796746 3647 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:33299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/krb5kdc/oryx_127.26.120.195.keytab
--principal=oryx/127.26.120.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.195:34353
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.120.195
--webserver_port=33405
--tserver_master_addrs=127.26.120.254:39675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3647
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.195
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 10:59:45.796939 3647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 10:59:45.797122 3647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 10:59:45.802568 3655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:45.802731 3652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 10:59:45.802706 3653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 10:59:45.802754 3647 server_base.cc:1034] running on GCE node
I20260929 10:59:45.803010 3647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 10:59:45.803267 3647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 10:59:45.804430 3647 hybrid_clock.cc:648] HybridClock initialized: now 1790679585804409 us; error 40 us; skew 500 ppm
Sep 29 10:59:45 dist-test-slave-2lzq krb5kdc[2779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679585, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.120.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 10:59:45.806893 3647 init.cc:377] Logged in from keytab as oryx/127.26.120.195@KRBTEST.COM (short username oryx)
I20260929 10:59:45.807335 3647 webserver.cc:460] Webserver started at http://127.26.120.195:33405/ using document root <none> and password file <none>
I20260929 10:59:45.807592 3647 fs_manager.cc:359] Metadata directory not provided
I20260929 10:59:45.807672 3647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 10:59:45.809120 3647 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:59:45.809759 3662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:45.809948 3647 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 10:59:45.810045 3647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-2/wal
uuid: "988bd6da9787461595fd445cb9cd21b5"
format_stamp: "Formatted at 2026-09-29 10:59:45 on dist-test-slave-2lzq"
server_key: "3542d4bc56b4157e10f343b0ecbec865"
server_key_iv: "fb38da7687886a463f7ea19102cbd3a5"
server_key_version: "encryptionkey@0"
I20260929 10:59:45.810422 3647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 10:59:45.833779 3647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 10:59:45.836124 3647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 10:59:45.836277 3647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 10:59:45.836529 3647 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 10:59:45.836879 3647 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 10:59:45.836913 3647 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:45.836936 3647 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 10:59:45.836980 3647 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 10:59:45.842804 3647 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.195:34353
I20260929 10:59:45.842847 3775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.195:34353 every 8 connection(s)
I20260929 10:59:45.843228 3647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
I20260929 10:59:45.847668 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 3647
Sep 29 10:59:45 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:45 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
W20260929 10:59:45.851466 3513 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:45.844069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:41891 (local address 127.26.120.254:39675)
0929 10:59:45.844144 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:45.844147 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:45.844327 (+ 180us) server_negotiation.cc:407] Connection header received
0929 10:59:45.844954 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:45.844955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:45.844965 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:45.845019 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:45.845954 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.846466 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.847515 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.847857 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.847951 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:45.851407 (+ 3456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.195:41891: BlockingRecv error: recv got EOF from 127.26.120.195:41891 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 10:59:45.851567 3778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:45.844037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.195:41891)
0929 10:59:45.844203 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:45.844240 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:45.844853 (+ 613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:45.845075 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:45.845078 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:45.845650 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:45.845843 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:45.845847 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.846557 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.846557 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:45.847353 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:45.847355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.847959 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.847961 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:45.848088 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:45.848737 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:45.848749 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:45.851347 (+ 2598us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":101,"threads_started":1}
W20260929 10:59:45.851890 3776 heartbeater.cc:644] Failed to heartbeat to 127.26.120.254:39675 (0 consecutive failures): Not authorized: Failed to ping master at 127.26.120.254:39675: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Sep 29 10:59:45 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:45.855997 3515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:45.848266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49126 (local address 127.26.120.254:39675)
0929 10:59:45.848321 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:45.848323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:45.848394 (+ 71us) server_negotiation.cc:407] Connection header received
0929 10:59:45.848449 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:45.848450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:45.848462 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:45.848529 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:45.849315 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.850050 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.851071 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.851269 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.851319 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:45.852865 (+ 1546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 10:59:45.852869 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 10:59:45.852870 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 10:59:45.852882 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 10:59:45.854509 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:45.854947 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:45.854950 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:45.854950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:45.855043 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 10:59:45.855247 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 10:59:45.855248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 10:59:45.855249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 10:59:45.855487 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 10:59:45.855682 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0929 10:59:45.855793 (+ 111us) server_negotiation.cc:299] Negotiation successful
0929 10:59:45.855909 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
Sep 29 10:59:45 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:45.857172 3778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:45.852139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.195:59545)
0929 10:59:45.852186 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:45.852195 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:45.852235 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:45.852370 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:45.852371 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:45.852505 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:45.852546 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:45.852547 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.853604 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.853605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:45.854152 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:45.854154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.854509 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.854510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:45.854582 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:45.855135 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:45.855139 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:45.857128 (+ 1989us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260929 10:59:45.857402 3513 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:45.852176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:59545 (local address 127.26.120.254:39675)
0929 10:59:45.852216 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:45.852218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:45.852227 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:59:45.852301 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:45.852302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:45.852310 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:45.852349 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:45.852622 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.853227 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.854245 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.854435 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.854509 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:45.857286 (+ 2777us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.195:59545: BlockingRecv error: recv got EOF from 127.26.120.195:59545 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Sep 29 10:59:45 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:45 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:45.861475 3778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:45.857624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.195:47197)
0929 10:59:45.857669 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:45.857679 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:45.857720 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:45.857831 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:45.857831 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:45.857932 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:45.857959 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:45.857960 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.858690 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.858691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:45.859161 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:45.859162 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.859498 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.859499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:45.859534 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:45.859932 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:45.859936 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:45.861429 (+ 1493us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260929 10:59:45.861641 3776 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 10:59:45.861693 3513 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:45.857694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:47197 (local address 127.26.120.254:39675)
0929 10:59:45.857727 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:45.857729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:45.857737 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:59:45.857758 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:45.857759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:45.857766 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:45.857826 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:45.858046 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.858617 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.859252 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:45.859432 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:45.859493 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:45.861646 (+ 2153us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.195:47197: BlockingRecv error: recv got EOF from 127.26.120.195:47197 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
Sep 29 10:59:46 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:46 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:46.598591 3780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:46.593838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.193:45775)
0929 10:59:46.594087 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:46.594098 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:46.594184 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:46.594354 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:46.594355 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:46.594566 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:46.594617 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:46.594619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:46.595287 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:46.595288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:46.595843 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:46.595844 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:46.596214 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:46.596215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:46.596278 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:46.596689 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:46.596693 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:46.598490 (+ 1797us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":116,"threads_started":1}
W20260929 10:59:46.598645 3781 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:46.593954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:45775 (local address 127.26.120.254:39675)
0929 10:59:46.594079 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:46.594081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:46.594169 (+ 88us) server_negotiation.cc:407] Connection header received
0929 10:59:46.594262 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:46.594263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:46.594275 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:46.594347 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:46.594699 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:46.595205 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:46.595929 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:46.596132 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:46.596213 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:46.598571 (+ 2358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.193:45775: BlockingRecv error: recv got EOF from 127.26.120.193:45775 (error 108)
Metrics: {"server-negotiator.queue_time_us":78,"thread_start_us":49,"threads_started":1}
Sep 29 10:59:46 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:46 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:46.725124 3782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:46.720358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.194:51849)
0929 10:59:46.720508 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:46.720519 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:46.720586 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:46.720710 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:46.720710 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:46.720923 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:46.720991 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:46.720993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:46.721862 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:46.721863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:46.722476 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:46.722478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:46.722838 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:46.722838 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:46.722905 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:46.723324 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:46.723328 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:46.725059 (+ 1731us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":76,"threads_started":1}
W20260929 10:59:46.725219 3781 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:46.720464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:51849 (local address 127.26.120.254:39675)
0929 10:59:46.720522 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:46.720524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:46.720580 (+ 56us) server_negotiation.cc:407] Connection header received
0929 10:59:46.720631 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:46.720632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:46.720642 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:46.720692 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:46.721074 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:46.721779 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:46.722557 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:46.722765 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:46.722840 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:46.725153 (+ 2313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.194:51849: BlockingRecv error: recv got EOF from 127.26.120.194:51849 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Sep 29 10:59:46 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:46 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:46.866390 3783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:46.862010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.195:59301)
0929 10:59:46.862170 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:46.862181 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:46.862229 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:46.862348 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:46.862349 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:46.862506 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:46.862555 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:46.862556 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:46.863167 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:46.863167 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:46.863745 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:46.863747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:46.864057 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:46.864057 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:46.864097 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:46.864512 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:46.864516 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:46.866321 (+ 1805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":109,"threads_started":1}
W20260929 10:59:46.866469 3781 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:46.862185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:59301 (local address 127.26.120.254:39675)
0929 10:59:46.862229 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:46.862231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:46.862241 (+ 10us) server_negotiation.cc:407] Connection header received
0929 10:59:46.862270 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:46.862271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:46.862281 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:46.862334 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:46.862620 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:46.863099 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:46.863818 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:46.863998 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:46.864050 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:46.866408 (+ 2358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.195:59301: BlockingRecv error: recv got EOF from 127.26.120.195:59301 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Sep 29 10:59:47 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:47 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:47.603578 3784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:47.599171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.193:41793)
0929 10:59:47.599299 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:47.599313 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:47.599363 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:47.599497 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:47.599498 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:47.599663 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:47.599717 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:47.599718 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:47.600464 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:47.600465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:47.600987 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:47.600988 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:47.601303 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:47.601304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:47.601344 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:47.601827 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:47.601831 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:47.603513 (+ 1682us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":79,"threads_started":1}
W20260929 10:59:47.603727 3785 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:47.599219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:41793 (local address 127.26.120.254:39675)
0929 10:59:47.599342 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:47.599344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:47.599355 (+ 11us) server_negotiation.cc:407] Connection header received
0929 10:59:47.599410 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:47.599411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:47.599422 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:47.599484 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:47.599779 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:47.600397 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:47.601052 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:47.601243 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:47.601291 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:47.603631 (+ 2340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.193:41793: BlockingRecv error: recv got EOF from 127.26.120.193:41793 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":73,"threads_started":1}
Sep 29 10:59:47 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:47 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:47.730129 3786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:47.725669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.194:54615)
0929 10:59:47.725812 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:47.725823 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:47.725880 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:47.726008 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:47.726009 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:47.726191 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:47.726243 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:47.726245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:47.726900 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:47.726900 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:47.727433 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:47.727434 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:47.727747 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:47.727748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:47.727789 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:47.728286 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:47.728291 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:47.730000 (+ 1709us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":81,"threads_started":1}
W20260929 10:59:47.730265 3785 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:47.725736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:54615 (local address 127.26.120.254:39675)
0929 10:59:47.725801 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:47.725803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:47.725885 (+ 82us) server_negotiation.cc:407] Connection header received
0929 10:59:47.725927 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:47.725928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:47.725938 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:47.725992 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:47.726312 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:47.726817 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:47.727503 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:47.727685 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:47.727734 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:47.730179 (+ 2445us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.194:54615: BlockingRecv error: recv got EOF from 127.26.120.194:54615 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 29 10:59:47 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:47 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:47.871078 3787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:47.866871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.195:36079)
0929 10:59:47.867005 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:47.867016 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:47.867067 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:47.867187 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:47.867187 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:47.867344 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:47.867402 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:47.867404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:47.868008 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:47.868009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:47.868526 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:47.868528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:47.868852 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:47.868853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:47.868915 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:47.869325 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:47.869330 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:47.871019 (+ 1689us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":58,"threads_started":1}
W20260929 10:59:47.871217 3785 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:47.867010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:36079 (local address 127.26.120.254:39675)
0929 10:59:47.867082 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:47.867084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:47.867093 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:59:47.867114 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:47.867115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:47.867125 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:47.867179 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:47.867466 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:47.867945 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:47.868591 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:47.868769 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:47.868819 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:47.871135 (+ 2316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.195:36079: BlockingRecv error: recv got EOF from 127.26.120.195:36079 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 29 10:59:48 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:48 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:48.613313 3788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:48.604012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.193:35785)
0929 10:59:48.604154 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:48.604165 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:48.604216 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:48.604336 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:48.604337 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:48.604486 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:48.604537 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:48.604538 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:48.605162 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:48.605162 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:48.605682 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:48.605683 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:48.606017 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:48.606018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:48.606082 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:48.611408 (+ 5326us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:48.611414 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:48.613204 (+ 1790us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":85,"threads_started":1}
W20260929 10:59:48.613361 3789 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:48.604094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:35785 (local address 127.26.120.254:39675)
0929 10:59:48.604214 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:48.604216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:48.604246 (+ 30us) server_negotiation.cc:407] Connection header received
0929 10:59:48.604265 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:48.604266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:48.604276 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:48.604323 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:48.604599 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:48.605099 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:48.605754 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:48.605954 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:48.606010 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:48.613286 (+ 7276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.193:35785: BlockingRecv error: recv got EOF from 127.26.120.193:35785 (error 108)
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":72,"threads_started":1}
Sep 29 10:59:48 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:48 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:48.735419 3790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:48.730611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.194:40525)
0929 10:59:48.730822 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:48.730834 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:48.730942 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:48.731093 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:48.731094 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:48.731287 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:48.731334 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:48.731342 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:48.732219 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:48.732220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:48.732756 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:48.732757 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:48.733092 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:48.733092 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:48.733134 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:48.733558 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:48.733562 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:48.735339 (+ 1777us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":100,"threads_started":1}
W20260929 10:59:48.735520 3789 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:48.730717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:40525 (local address 127.26.120.254:39675)
0929 10:59:48.730807 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:48.730810 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:48.730914 (+ 104us) server_negotiation.cc:407] Connection header received
0929 10:59:48.731000 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:48.731001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:48.731015 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:48.731086 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:48.731418 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:48.732130 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:48.732825 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:48.733021 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:48.733082 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:48.735429 (+ 2347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.194:40525: BlockingRecv error: recv got EOF from 127.26.120.194:40525 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 29 10:59:48 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:48 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:48.876188 3791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:48.871529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.195:46941)
0929 10:59:48.871721 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:48.871734 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:48.871812 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:48.871979 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:48.871980 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:48.872150 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:48.872197 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:48.872199 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:48.872873 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:48.872874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:48.873395 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:48.873397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:48.873760 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:48.873761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:48.873818 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:48.874277 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:48.874282 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:48.876082 (+ 1800us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":99,"threads_started":1}
W20260929 10:59:48.876226 3789 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:48.871610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:46941 (local address 127.26.120.254:39675)
0929 10:59:48.871705 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:48.871707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:48.871790 (+ 83us) server_negotiation.cc:407] Connection header received
0929 10:59:48.871876 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:48.871877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:48.871895 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:48.871970 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:48.872273 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:48.872783 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:48.873485 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:48.873675 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:48.873760 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:48.876162 (+ 2402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.195:46941: BlockingRecv error: recv got EOF from 127.26.120.195:46941 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 29 10:59:49 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:49 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:49.618433 3792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:49.613786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.193:46143)
0929 10:59:49.613967 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:49.613978 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:49.614042 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:49.614282 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:49.614283 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:49.614474 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:49.614527 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:49.614529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:49.615159 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:49.615160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:49.615707 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:49.615709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:49.616033 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:49.616034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:49.616073 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:49.616501 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:49.616505 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:49.618329 (+ 1824us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":100,"threads_started":1}
W20260929 10:59:49.618515 3793 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:49.613847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:46143 (local address 127.26.120.254:39675)
0929 10:59:49.613976 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:49.613978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:49.614039 (+ 61us) server_negotiation.cc:407] Connection header received
0929 10:59:49.614170 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:49.614171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:49.614181 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:49.614241 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:49.614594 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:49.615094 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:49.615776 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:49.615973 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:49.616024 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:49.618431 (+ 2407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.193:46143: BlockingRecv error: recv got EOF from 127.26.120.193:46143 (error 108)
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":46,"threads_started":1}
Sep 29 10:59:49 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:49 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:49.740662 3794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:49.735864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.194:41265)
0929 10:59:49.736057 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:49.736069 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:49.736160 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:49.736324 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:49.736325 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:49.736505 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:49.736555 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:49.736557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:49.737248 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:49.737249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:49.737824 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:49.737826 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:49.738252 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:49.738253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:49.738315 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:49.738745 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:49.738750 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:49.740604 (+ 1854us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":108,"threads_started":1}
W20260929 10:59:49.740767 3793 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:49.735960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:41265 (local address 127.26.120.254:39675)
0929 10:59:49.736050 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:49.736052 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:49.736140 (+ 88us) server_negotiation.cc:407] Connection header received
0929 10:59:49.736242 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:49.736243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:49.736259 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:49.736318 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:49.736635 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:49.737162 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:49.737918 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:49.738169 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:49.738251 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:49.740695 (+ 2444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.194:41265: BlockingRecv error: recv got EOF from 127.26.120.194:41265 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 29 10:59:49 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:49 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:49.881635 3795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:49.876824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.195:60233)
0929 10:59:49.876999 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:49.877013 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:49.877114 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:49.877275 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:49.877277 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:49.877460 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:49.877510 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:49.877512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:49.878217 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:49.878218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:49.878809 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:49.878811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:49.879185 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:49.879185 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:49.879244 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:49.879746 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:49.879751 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:49.881539 (+ 1788us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":99,"threads_started":1}
W20260929 10:59:49.881688 3793 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:49.876928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:60233 (local address 127.26.120.254:39675)
0929 10:59:49.877002 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:49.877005 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:49.877093 (+ 88us) server_negotiation.cc:407] Connection header received
0929 10:59:49.877187 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:49.877188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:49.877203 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:49.877270 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:49.877591 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:49.878139 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:49.878897 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:49.879100 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:49.879183 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:49.881623 (+ 2440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.195:60233: BlockingRecv error: recv got EOF from 127.26.120.195:60233 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 29 10:59:50 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:50 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:50.623477 3796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:50.618879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.193:41669)
0929 10:59:50.619042 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:50.619054 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:50.619111 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:50.619243 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:50.619244 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:50.619408 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:50.619462 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:50.619464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:50.620082 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:50.620082 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:50.620612 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:50.620614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:50.620928 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:50.620928 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:50.620973 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:50.621426 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:50.621431 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:50.623379 (+ 1948us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":104,"threads_started":1}
W20260929 10:59:50.623540 3797 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:50.618992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:41669 (local address 127.26.120.254:39675)
0929 10:59:50.619101 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:50.619103 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:50.619112 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:59:50.619169 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:50.619170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:50.619180 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:50.619229 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:50.619525 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:50.620014 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:50.620677 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:50.620865 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:50.620913 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:50.623468 (+ 2555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.193:41669: BlockingRecv error: recv got EOF from 127.26.120.193:41669 (error 108)
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":54,"threads_started":1}
Sep 29 10:59:50 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:50 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:50.745991 3798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:50.741184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.194:43235)
0929 10:59:50.741352 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:50.741369 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:50.741437 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:50.741571 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:50.741572 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:50.741731 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:50.741780 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:50.741782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:50.742493 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:50.742493 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:50.743006 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:50.743007 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:50.743377 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:50.743378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:50.743422 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:50.743870 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:50.743874 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:50.745933 (+ 2059us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":105,"threads_started":1}
W20260929 10:59:50.746150 3797 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:50.741344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:43235 (local address 127.26.120.254:39675)
0929 10:59:50.741405 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:50.741407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:50.741423 (+ 16us) server_negotiation.cc:407] Connection header received
0929 10:59:50.741494 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:50.741494 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:50.741505 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:50.741558 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:50.741837 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:50.742430 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:50.743092 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:50.743315 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:50.743367 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:50.746027 (+ 2660us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.194:43235: BlockingRecv error: recv got EOF from 127.26.120.194:43235 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 29 10:59:50 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:50 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:50.886397 3799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:50.882143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.195:51233)
0929 10:59:50.882291 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:50.882300 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:50.882356 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:50.882482 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:50.882483 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:50.882658 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:50.882707 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:50.882709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:50.883332 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:50.883333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:50.883852 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:50.883854 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:50.884181 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:50.884182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:50.884221 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:50.884666 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:50.884671 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:50.886337 (+ 1666us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":83,"threads_started":1}
W20260929 10:59:50.886507 3797 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:50.882172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:51233 (local address 127.26.120.254:39675)
0929 10:59:50.882231 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:50.882233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:50.882352 (+ 119us) server_negotiation.cc:407] Connection header received
0929 10:59:50.882412 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:50.882412 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:50.882426 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:50.882467 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:50.882767 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:50.883264 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:50.883933 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:50.884124 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:50.884171 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:50.886424 (+ 2253us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.195:51233: BlockingRecv error: recv got EOF from 127.26.120.195:51233 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 29 10:59:51 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:51 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:51.628947 3800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:51.624036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.193:37643)
0929 10:59:51.624186 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:51.624201 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:51.624269 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:51.624451 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:51.624452 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:51.624614 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:51.624666 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:51.624668 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:51.625623 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:51.625624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:51.626199 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:51.626201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:51.626585 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:51.626585 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:51.626626 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:51.627082 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:51.627087 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:51.628889 (+ 1802us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":75,"threads_started":1}
W20260929 10:59:51.629088 3801 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:51.624100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:37643 (local address 127.26.120.254:39675)
0929 10:59:51.624247 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:51.624249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:51.624258 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:59:51.624368 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:51.624369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:51.624379 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:51.624432 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:51.624737 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:51.625538 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:51.626273 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:51.626508 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:51.626576 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:51.628995 (+ 2419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.193:37643: BlockingRecv error: recv got EOF from 127.26.120.193:37643 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":82,"threads_started":1}
Sep 29 10:59:51 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:51 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:51.751152 3802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:51.746438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.194:52325)
0929 10:59:51.746627 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:51.746640 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:51.746737 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:51.746909 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:51.746910 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:51.747098 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:51.747159 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:51.747161 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:51.747857 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:51.747858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:51.748385 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:51.748386 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:51.748756 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:51.748757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:51.748827 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:51.749244 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:51.749249 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:51.751055 (+ 1806us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":87,"threads_started":1}
W20260929 10:59:51.751205 3801 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:51.746521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:52325 (local address 127.26.120.254:39675)
0929 10:59:51.746610 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:51.746613 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:51.746715 (+ 102us) server_negotiation.cc:407] Connection header received
0929 10:59:51.746820 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:51.746821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:51.746833 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:51.746902 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:51.747247 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:51.747768 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:51.748472 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:51.748672 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:51.748755 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:51.751140 (+ 2385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.194:52325: BlockingRecv error: recv got EOF from 127.26.120.194:52325 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 29 10:59:51 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:51 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:51.891474 3803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:51.886800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.195:40293)
0929 10:59:51.886977 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:51.886991 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:51.887091 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:51.887245 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:51.887246 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:51.887444 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:51.887491 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:51.887492 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:51.888145 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:51.888146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:51.888724 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:51.888726 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:51.889099 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:51.889100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:51.889162 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:51.889682 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:51.889687 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:51.891377 (+ 1690us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":76,"threads_started":1}
W20260929 10:59:51.891525 3801 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:51.886896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:40293 (local address 127.26.120.254:39675)
0929 10:59:51.886988 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:51.886991 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:51.887069 (+ 78us) server_negotiation.cc:407] Connection header received
0929 10:59:51.887156 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:51.887157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:51.887174 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:51.887238 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:51.887569 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:51.888063 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:51.888813 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:51.889005 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:51.889099 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:51.891460 (+ 2361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.195:40293: BlockingRecv error: recv got EOF from 127.26.120.195:40293 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 29 10:59:52 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:52 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:52.634378 3804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:52.629552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.193:56593)
0929 10:59:52.629675 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:52.629691 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:52.629742 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:52.629895 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:52.629895 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:52.630108 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:52.630165 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:52.630167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:52.630835 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:52.630836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:52.631391 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:52.631393 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:52.631932 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:52.631933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:52.632010 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:52.632460 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:52.632465 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:52.634319 (+ 1854us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":69,"threads_started":1}
W20260929 10:59:52.634513 3805 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:52.629606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:56593 (local address 127.26.120.254:39675)
0929 10:59:52.629745 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:52.629747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:52.629758 (+ 11us) server_negotiation.cc:407] Connection header received
0929 10:59:52.629806 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:52.629806 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:52.629817 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:52.629871 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:52.630244 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:52.630746 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:52.631525 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:52.631827 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:52.631911 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:52.634419 (+ 2508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.193:56593: BlockingRecv error: recv got EOF from 127.26.120.193:56593 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":82,"threads_started":1}
Sep 29 10:59:52 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:52 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:52.756115 3806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:52.751673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.194:45071)
0929 10:59:52.751803 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:52.751814 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:52.751887 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:52.752007 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:52.752008 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:52.752172 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:52.752220 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:52.752222 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:52.752834 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:52.752835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:52.753351 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:52.753353 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:52.753699 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:52.753700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:52.753741 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:52.754207 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:52.754212 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:52.756056 (+ 1844us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":81,"threads_started":1}
W20260929 10:59:52.756244 3805 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:52.751729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:45071 (local address 127.26.120.254:39675)
0929 10:59:52.751801 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:52.751803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:52.751875 (+ 72us) server_negotiation.cc:407] Connection header received
0929 10:59:52.751932 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:52.751933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:52.751943 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:52.751994 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:52.752281 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:52.752770 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:52.753455 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:52.753637 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:52.753686 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:52.756164 (+ 2478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.194:45071: BlockingRecv error: recv got EOF from 127.26.120.194:45071 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 29 10:59:52 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:52 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:52.896638 3807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:52.892027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.195:56871)
0929 10:59:52.892183 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:52.892195 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:52.892259 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:52.892433 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:52.892434 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:52.892611 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:52.892658 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:52.892660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:52.893317 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:52.893318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:52.893845 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:52.893846 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:52.894297 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:52.894298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:52.894365 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:52.894780 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:52.894784 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:52.896579 (+ 1795us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":87,"threads_started":1}
W20260929 10:59:52.896752 3805 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:52.892068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:56871 (local address 127.26.120.254:39675)
0929 10:59:52.892156 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:52.892158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:52.892264 (+ 106us) server_negotiation.cc:407] Connection header received
0929 10:59:52.892309 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:52.892310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:52.892324 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:52.892417 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:52.892738 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:52.893235 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:52.893961 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:52.894215 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:52.894295 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:52.896675 (+ 2380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.195:56871: BlockingRecv error: recv got EOF from 127.26.120.195:56871 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 29 10:59:53 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:53 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:53.645193 3808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:53.634876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.193:41771)
0929 10:59:53.635019 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:53.635037 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:53.635088 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:53.635317 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:53.635317 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:53.635488 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:53.635538 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:53.635540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:53.636198 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:53.636199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:53.636778 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:53.636781 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:53.637157 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:53.637158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:53.637219 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:53.643055 (+ 5836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:53.643060 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:53.645085 (+ 2025us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":86,"threads_started":1}
W20260929 10:59:53.645289 3809 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:53.635057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:41771 (local address 127.26.120.254:39675)
0929 10:59:53.635192 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:53.635194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:53.635201 (+ 7us) server_negotiation.cc:407] Connection header received
0929 10:59:53.635224 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:53.635224 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:53.635234 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:53.635298 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:53.635606 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:53.636107 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:53.636870 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:53.637068 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:53.637126 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:53.645207 (+ 8081us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.193:41771: BlockingRecv error: recv got EOF from 127.26.120.193:41771 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":80,"threads_started":1}
Sep 29 10:59:53 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:53 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:53.761178 3810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:53.756600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.194:47175)
0929 10:59:53.756748 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:53.756764 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:53.756812 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:53.756979 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:53.756980 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:53.757140 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:53.757189 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:53.757191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:53.757963 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:53.757964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:53.758549 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:53.758551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:53.758858 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:53.758859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:53.758900 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:53.759338 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:53.759343 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:53.761118 (+ 1775us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":92,"threads_started":1}
W20260929 10:59:53.761304 3809 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:53.756776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:47175 (local address 127.26.120.254:39675)
0929 10:59:53.756847 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:53.756849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:53.756857 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:59:53.756883 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:53.756884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:53.756894 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:53.756955 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:53.757254 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:53.757884 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:53.758616 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:53.758799 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:53.758848 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:53.761226 (+ 2378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.194:47175: BlockingRecv error: recv got EOF from 127.26.120.194:47175 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 29 10:59:53 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:53 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:53.901695 3811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:53.897097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.195:36101)
0929 10:59:53.897258 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:53.897270 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:53.897369 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:53.897562 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:53.897562 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:53.897739 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:53.897785 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:53.897786 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:53.898520 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:53.898520 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:53.899052 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:53.899054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:53.899404 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:53.899405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:53.899472 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:53.899902 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:53.899906 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:53.901635 (+ 1729us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":82,"threads_started":1}
W20260929 10:59:53.901851 3809 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:53.897183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:36101 (local address 127.26.120.254:39675)
0929 10:59:53.897255 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:53.897257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:53.897349 (+ 92us) server_negotiation.cc:407] Connection header received
0929 10:59:53.897463 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:53.897464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:53.897479 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:53.897539 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:53.897862 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:53.898439 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:53.899138 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:53.899333 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:53.899406 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:53.901761 (+ 2355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.195:36101: BlockingRecv error: recv got EOF from 127.26.120.195:36101 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 29 10:59:54 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:54 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:54.650550 3812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:54.645692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.193:41769)
0929 10:59:54.645852 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:54.645864 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:54.645920 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:54.646043 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:54.646043 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:54.646255 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:54.646307 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:54.646309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:54.647000 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:54.647001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:54.647571 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:54.647573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:54.647930 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:54.647931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:54.647983 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:54.648566 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:54.648570 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:54.650489 (+ 1919us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":83,"threads_started":1}
W20260929 10:59:54.650687 3813 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:54.645798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:41769 (local address 127.26.120.254:39675)
0929 10:59:54.645926 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:54.645928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:54.645936 (+ 8us) server_negotiation.cc:407] Connection header received
0929 10:59:54.645959 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:54.645960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:54.645970 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:54.646028 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:54.646375 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:54.646920 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:54.647644 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:54.647833 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:54.647885 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:54.650599 (+ 2714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.193:41769: BlockingRecv error: recv got EOF from 127.26.120.193:41769 (error 108)
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":77,"threads_started":1}
Sep 29 10:59:54 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:54 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:54.766314 3814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:54.761682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.194:42191)
0929 10:59:54.761850 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:54.761860 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:54.761918 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:54.762035 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:54.762035 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:54.762220 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:54.762270 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:54.762272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:54.762890 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:54.762890 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:54.763465 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:54.763468 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:54.763810 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:54.763811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:54.763874 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:54.764427 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:54.764431 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:54.766253 (+ 1822us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":99,"threads_started":1}
W20260929 10:59:54.766396 3813 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:54.761740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:42191 (local address 127.26.120.254:39675)
0929 10:59:54.761803 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:54.761805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:54.761916 (+ 111us) server_negotiation.cc:407] Connection header received
0929 10:59:54.761975 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:54.761975 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:54.761987 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:54.762023 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:54.762325 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:54.762826 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:54.763540 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:54.763747 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:54.763801 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:54.766334 (+ 2533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.194:42191: BlockingRecv error: recv got EOF from 127.26.120.194:42191 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Sep 29 10:59:54 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:54 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:54.907091 3815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:54.902166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.195:55319)
0929 10:59:54.902332 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:54.902345 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:54.902431 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:54.902583 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:54.902584 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:54.902774 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:54.902840 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:54.902842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:54.903656 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:54.903657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:54.904198 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:54.904200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:54.904565 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:54.904566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:54.904626 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:54.905112 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:54.905123 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:54.907031 (+ 1908us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":96,"threads_started":1}
W20260929 10:59:54.907270 3813 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:54.902276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:55319 (local address 127.26.120.254:39675)
0929 10:59:54.902336 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:54.902339 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:54.902408 (+ 69us) server_negotiation.cc:407] Connection header received
0929 10:59:54.902497 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:54.902498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:54.902511 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:54.902576 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:54.902926 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:54.903530 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:54.904282 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:54.904480 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:54.904563 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:54.907206 (+ 2643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.195:55319: BlockingRecv error: recv got EOF from 127.26.120.195:55319 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Sep 29 10:59:55 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:55 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:55.655849 3816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:55.651095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.193:46223)
0929 10:59:55.651242 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:55.651253 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:55.651303 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:55.651459 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:55.651460 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:55.651638 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:55.651688 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:55.651690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:55.652490 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:55.652491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:55.653028 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:55.653029 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:55.653396 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:55.653396 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:55.653461 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:55.653915 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:55.653920 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:55.655752 (+ 1832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":83,"threads_started":1}
W20260929 10:59:55.655906 3817 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:55.651191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:46223 (local address 127.26.120.254:39675)
0929 10:59:55.651314 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:55.651316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:55.651325 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:59:55.651351 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:55.651351 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:55.651362 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:55.651445 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:55.651781 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:55.652407 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:55.653124 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:55.653325 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:55.653397 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:55.655833 (+ 2436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.193:46223: BlockingRecv error: recv got EOF from 127.26.120.193:46223 (error 108)
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":67,"threads_started":1}
Sep 29 10:59:55 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:55 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:55.771606 3818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:55.766817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.194:46547)
0929 10:59:55.766943 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:55.766958 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:55.767015 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:55.767150 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:55.767151 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:55.767314 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:55.767369 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:55.767370 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:55.768000 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:55.768002 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:55.768668 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:55.768671 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:55.768997 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:55.768997 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:55.769038 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:55.769528 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:55.769533 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:55.771498 (+ 1965us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
W20260929 10:59:55.771637 3817 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:55.766885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:46547 (local address 127.26.120.254:39675)
0929 10:59:55.766962 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:55.766964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:55.767016 (+ 52us) server_negotiation.cc:407] Connection header received
0929 10:59:55.767068 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:55.767069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:55.767079 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:55.767138 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:55.767430 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:55.767914 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:55.768743 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:55.768932 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:55.768988 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:55.771578 (+ 2590us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.194:46547: BlockingRecv error: recv got EOF from 127.26.120.194:46547 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 29 10:59:55 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:55 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:55.912524 3819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:55.907644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.195:42903)
0929 10:59:55.907812 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:55.907828 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:55.907884 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:55.908015 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:55.908015 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:55.908173 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:55.908220 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:55.908221 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:55.909151 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:55.909153 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:55.909678 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:55.909679 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:55.910126 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:55.910127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:55.910169 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:55.910631 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:55.910635 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:55.912456 (+ 1821us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":72,"threads_started":1}
W20260929 10:59:55.912652 3817 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:55.907836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:42903 (local address 127.26.120.254:39675)
0929 10:59:55.907904 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:55.907905 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:55.907914 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:59:55.907936 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:55.907937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:55.907947 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:55.907995 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:55.908282 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:55.909059 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:55.909794 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:55.910005 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:55.910123 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:55.912554 (+ 2431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.195:42903: BlockingRecv error: recv got EOF from 127.26.120.195:42903 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 29 10:59:56 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:56 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:56.660816 3820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:56.656340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.193:60627)
0929 10:59:56.656495 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:56.656506 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:56.656558 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:56.656713 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:56.656714 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:56.656879 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:56.656927 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:56.656928 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:56.657562 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:56.657562 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:56.658156 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:56.658158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:56.658470 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:56.658470 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:56.658511 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:56.658936 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:56.658940 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:56.660747 (+ 1807us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":84,"threads_started":1}
W20260929 10:59:56.660971 3821 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:56.656463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:60627 (local address 127.26.120.254:39675)
0929 10:59:56.656583 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:56.656585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:56.656594 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:59:56.656625 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:56.656626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:56.656635 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:56.656694 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:56.656993 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:56.657495 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:56.658224 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:56.658410 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:56.658468 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:56.660841 (+ 2373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.193:60627: BlockingRecv error: recv got EOF from 127.26.120.193:60627 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":71,"threads_started":1}
Sep 29 10:59:56 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:56 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:56.776798 3822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:56.772104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.194:33087)
0929 10:59:56.772261 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:56.772272 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:56.772349 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:56.772507 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:56.772508 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:56.772697 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:56.772742 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:56.772744 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:56.773418 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:56.773419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:56.773953 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:56.773954 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:56.774396 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:56.774397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:56.774470 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:56.775070 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:56.775075 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:56.776727 (+ 1652us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":78,"threads_started":1}
W20260929 10:59:56.776886 3821 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:56.772153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:33087 (local address 127.26.120.254:39675)
0929 10:59:56.772216 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:56.772219 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:56.772337 (+ 118us) server_negotiation.cc:407] Connection header received
0929 10:59:56.772411 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:56.772412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:56.772425 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:56.772498 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:56.772824 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:56.773329 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:56.774040 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:56.774278 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:56.774367 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:56.776820 (+ 2453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.194:33087: BlockingRecv error: recv got EOF from 127.26.120.194:33087 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Sep 29 10:59:56 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:56 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:56.917843 3823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:56.912923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.195:60255)
0929 10:59:56.913107 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:56.913118 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:56.913205 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:56.913362 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:56.913363 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:56.913612 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:56.913658 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:56.913660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:56.914375 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:56.914376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:56.914921 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:56.914923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:56.915301 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:56.915302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:56.915360 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:56.915851 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:56.915855 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:56.917774 (+ 1919us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":135,"threads_started":1}
W20260929 10:59:56.917937 3821 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:56.913014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:60255 (local address 127.26.120.254:39675)
0929 10:59:56.913094 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:56.913097 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:56.913184 (+ 87us) server_negotiation.cc:407] Connection header received
0929 10:59:56.913269 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:56.913270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:56.913290 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:56.913356 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:56.913734 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:56.914295 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:56.915003 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:56.915194 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:56.915300 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:56.917855 (+ 2555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.195:60255: BlockingRecv error: recv got EOF from 127.26.120.195:60255 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 29 10:59:57 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:57 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:57.665743 3824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:57.661325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.193:38047)
0929 10:59:57.661461 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:57.661473 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:57.661533 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:57.661743 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:57.661744 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:57.661954 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:57.662005 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:57.662007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:57.662690 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:57.662691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:57.663201 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:57.663203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:57.663521 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:57.663522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:57.663573 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:57.664013 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:57.664018 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:57.665685 (+ 1667us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":84,"threads_started":1}
W20260929 10:59:57.665858 3825 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:57.661380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:38047 (local address 127.26.120.254:39675)
0929 10:59:57.661497 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:57.661499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:57.661513 (+ 14us) server_negotiation.cc:407] Connection header received
0929 10:59:57.661619 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:57.661620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:57.661635 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:57.661730 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:57.662108 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:57.662621 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:57.663267 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:57.663459 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:57.663507 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:57.665774 (+ 2267us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.193:38047: BlockingRecv error: recv got EOF from 127.26.120.193:38047 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":57,"threads_started":1}
Sep 29 10:59:57 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:57 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:57.781774 3826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:57.777254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.194:34199)
0929 10:59:57.777403 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:57.777416 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:57.777474 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:57.777605 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:57.777606 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:57.777767 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:57.777814 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:57.777815 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:57.778470 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:57.778472 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:57.778991 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:57.778993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:57.779317 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:57.779318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:57.779358 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:57.779801 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:57.779805 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:57.781679 (+ 1874us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":89,"threads_started":1}
W20260929 10:59:57.781831 3825 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:57.777341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:34199 (local address 127.26.120.254:39675)
0929 10:59:57.777417 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:57.777419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:57.777462 (+ 43us) server_negotiation.cc:407] Connection header received
0929 10:59:57.777521 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:57.777521 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:57.777532 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:57.777585 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:57.777876 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:57.778396 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:57.779074 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:57.779256 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:57.779304 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:57.781770 (+ 2466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.194:34199: BlockingRecv error: recv got EOF from 127.26.120.194:34199 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 29 10:59:57 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:57 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:57.923177 3827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:57.918336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.195:50645)
0929 10:59:57.918473 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:57.918488 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:57.918566 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:57.918711 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:57.918712 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:57.918897 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:57.918943 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:57.918945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:57.919664 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:57.919665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:57.920209 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:57.920211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:57.920598 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:57.920599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:57.920672 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:57.921120 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:57.921124 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:57.923081 (+ 1957us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":66,"threads_started":1}
W20260929 10:59:57.923230 3825 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:57.918436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:50645 (local address 127.26.120.254:39675)
0929 10:59:57.918513 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:57.918515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:57.918558 (+ 43us) server_negotiation.cc:407] Connection header received
0929 10:59:57.918635 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:57.918636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:57.918650 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:57.918694 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:57.919025 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:57.919571 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:57.920308 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:57.920505 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:57.920585 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:57.923165 (+ 2580us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.195:50645: BlockingRecv error: recv got EOF from 127.26.120.195:50645 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 29 10:59:58 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:58 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:58.676174 3828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:58.666291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.193:59733)
0929 10:59:58.666458 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:58.666469 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:58.666523 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:58.666678 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:58.666679 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:58.666836 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:58.666885 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:58.666887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:58.667534 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:58.667534 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:58.668073 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:58.668074 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:58.668487 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:58.668488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:58.668528 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:58.674161 (+ 5633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:58.674167 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:58.676103 (+ 1936us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":86,"threads_started":1}
W20260929 10:59:58.676311 3829 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:58.666367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:59733 (local address 127.26.120.254:39675)
0929 10:59:58.666485 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:58.666493 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:58.666510 (+ 17us) server_negotiation.cc:407] Connection header received
0929 10:59:58.666588 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:58.666588 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:58.666599 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:58.666660 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:58.666946 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:58.667463 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:58.668175 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:58.668414 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:58.668478 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:58.676221 (+ 7743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.193:59733: BlockingRecv error: recv got EOF from 127.26.120.193:59733 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":65,"threads_started":1}
Sep 29 10:59:58 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:58 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:58.786881 3830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:58.782285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.194:57385)
0929 10:59:58.782456 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:58.782470 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:58.782540 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:58.782754 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:58.782754 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:58.782913 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:58.782958 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:58.782960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:58.783627 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:58.783628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:58.784151 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:58.784152 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:58.784542 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:58.784542 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:58.784582 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:58.785085 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:58.785090 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:58.786824 (+ 1734us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":80,"threads_started":1}
W20260929 10:59:58.786985 3829 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:58.782450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:57385 (local address 127.26.120.254:39675)
0929 10:59:58.782571 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:58.782620 (+ 49us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:58.782632 (+ 12us) server_negotiation.cc:407] Connection header received
0929 10:59:58.782663 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:58.782663 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:58.782681 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:58.782740 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:58.783047 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:58.783538 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:58.784260 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:58.784472 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:58.784531 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:58.786918 (+ 2387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.194:57385: BlockingRecv error: recv got EOF from 127.26.120.194:57385 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Sep 29 10:59:58 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:58 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:58.928566 3831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:58.923691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.195:55803)
0929 10:59:58.923879 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:58.923892 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:58.923988 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:58.924137 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:58.924138 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:58.924328 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:58.924374 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:58.924376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:58.925071 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:58.925072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:58.925623 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:58.925624 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:58.926026 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:58.926027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:58.926126 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:58.926625 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:58.926629 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:58.928498 (+ 1869us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":91,"threads_started":1}
W20260929 10:59:58.928695 3829 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:58.923795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:55803 (local address 127.26.120.254:39675)
0929 10:59:58.923883 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:58.923885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:58.923970 (+ 85us) server_negotiation.cc:407] Connection header received
0929 10:59:58.924043 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:58.924044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:58.924059 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:58.924131 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:58.924470 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:58.924988 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:58.925739 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:58.925940 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:58.926024 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:58.928596 (+ 2572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.195:55803: BlockingRecv error: recv got EOF from 127.26.120.195:55803 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 29 10:59:59 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:59 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:59.681492 3832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:59.676745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.193:59025)
0929 10:59:59.676937 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:59.676948 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:59.677005 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:59.677145 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:59.677146 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:59.677328 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:59.677526 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:59.677530 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:59.678216 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:59.678217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:59.678777 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:59.678779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:59.679128 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:59.679129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:59.679170 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:59.679597 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:59.679601 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:59.681434 (+ 1833us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":88,"threads_started":1}
W20260929 10:59:59.681643 3833 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:59.676864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:59025 (local address 127.26.120.254:39675)
0929 10:59:59.676985 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:59.676987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:59.676998 (+ 11us) server_negotiation.cc:407] Connection header received
0929 10:59:59.677054 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:59.677054 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:59.677064 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:59.677124 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:59.677614 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:59.678145 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:59.678860 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:59.679062 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:59.679119 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:59.681552 (+ 2433us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.193:59025: BlockingRecv error: recv got EOF from 127.26.120.193:59025 (error 108)
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":72,"threads_started":1}
Sep 29 10:59:59 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:59 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:59.792160 3834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:59.787399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.194:38501)
0929 10:59:59.787579 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:59.787606 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:59.787664 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:59.787783 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:59.787784 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:59.787950 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:59.787998 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:59.787999 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:59.788689 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:59.788690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:59.789219 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:59.789221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:59.789597 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:59.789598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:59.789636 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:59.790093 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:59.790097 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:59.792093 (+ 1996us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":81,"threads_started":1}
W20260929 10:59:59.792265 3833 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:59.787594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:38501 (local address 127.26.120.254:39675)
0929 10:59:59.787658 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:59.787660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:59.787669 (+ 9us) server_negotiation.cc:407] Connection header received
0929 10:59:59.787719 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:59.787719 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:59.787729 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:59.787777 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:59.788081 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:59.788613 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:59.789294 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:59.789526 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:59.789588 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:59.792193 (+ 2605us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.194:38501: BlockingRecv error: recv got EOF from 127.26.120.194:38501 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Sep 29 10:59:59 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 10:59:59 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.195@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 10:59:59.934317 3835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 10:59:59.929273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.195:56201)
0929 10:59:59.929651 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0929 10:59:59.929664 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 10:59:59.929758 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 10:59:59.929926 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 10:59:59.929927 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 10:59:59.930185 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0929 10:59:59.930234 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:59.930236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:59.930935 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:59.930936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:59.931555 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 10:59:59.931557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 10:59:59.931998 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 10:59:59.931999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 10:59:59.932060 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:59.932517 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 10:59:59.932521 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 10:59:59.934258 (+ 1737us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":199,"threads_started":1}
W20260929 10:59:59.934474 3833 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 10:59:59.929431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:56201 (local address 127.26.120.254:39675)
0929 10:59:59.929622 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0929 10:59:59.929625 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 10:59:59.929738 (+ 113us) server_negotiation.cc:407] Connection header received
0929 10:59:59.929824 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 10:59:59.929825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 10:59:59.929839 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 10:59:59.929916 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 10:59:59.930314 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:59.930823 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:59.931646 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 10:59:59.931927 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 10:59:59.931981 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 10:59:59.934383 (+ 2402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.195:56201: BlockingRecv error: recv got EOF from 127.26.120.195:56201 (error 108)
Metrics: {"server-negotiator.queue_time_us":160}
Sep 29 11:00:00 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 11:00:00 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.193@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 11:00:00.686513 3836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:00.682018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.193:39437)
0929 11:00:00.682161 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0929 11:00:00.682173 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 11:00:00.682233 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:00:00.682402 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:00:00.682403 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:00:00.682597 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:00:00.682665 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:00.682668 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:00.683377 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:00.683377 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:00.683902 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:00.683904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:00.684240 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:00.684241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:00.684284 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:00.684721 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:00:00.684726 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:00:00.686454 (+ 1728us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":88,"threads_started":1}
W20260929 11:00:00.686596 3837 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 11:00:00.682090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:39437 (local address 127.26.120.254:39675)
0929 11:00:00.682209 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0929 11:00:00.682211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:00:00.682219 (+ 8us) server_negotiation.cc:407] Connection header received
0929 11:00:00.682304 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:00:00.682305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:00:00.682315 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:00:00.682363 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:00:00.682748 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:00.683305 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:00.683981 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:00.684173 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:00.684227 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:00.686543 (+ 2316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.193:39437: BlockingRecv error: recv got EOF from 127.26.120.193:39437 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":60,"threads_started":1}
Sep 29 11:00:00 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 11:00:00 dist-test-slave-2lzq krb5kdc[2779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.120.194@KRBTEST.COM for oryx/127.26.120.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 11:00:00.796880 3838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:00.792657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39675 (local address 127.26.120.194:41281)
0929 11:00:00.792798 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0929 11:00:00.792808 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 11:00:00.792862 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:00:00.792974 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:00:00.792975 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:00:00.793136 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:00:00.793184 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:00.793185 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:00.793798 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:00.793799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:00.794336 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:00.794338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:00.794671 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:00.794671 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:00.794711 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:00.795139 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:00:00.795143 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:00:00.796800 (+ 1657us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.120.254:39675: Server oryx/127.26.120.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":82,"threads_started":1}
W20260929 11:00:00.797014 3837 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 11:00:00.792728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:41281 (local address 127.26.120.254:39675)
0929 11:00:00.792780 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0929 11:00:00.792781 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 11:00:00.792861 (+ 80us) server_negotiation.cc:407] Connection header received
0929 11:00:00.792909 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:00:00.792909 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:00:00.792920 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:00:00.792968 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:00:00.793243 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:00.793732 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:00.794406 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:00.794594 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:00.794655 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:00.796914 (+ 2259us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.120.194:41281: BlockingRecv error: recv got EOF from 127.26.120.194:41281 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 11:00:00.854389 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 3362
I20260929 11:00:00.858666 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 3514
I20260929 11:00:00.862637 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 3647
I20260929 11:00:00.866839 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 3292
2026-09-29T11:00:00Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17502 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-29T11:00:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T11:00:00Z Disabled control of system clock
I20260929 11:00:00.894786 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:43313
--webserver_interface=127.26.120.254
--webserver_port=0
--builtin_ntp_servers=127.26.120.212:33117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:43313
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260929 11:00:00.973784 3842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:00:00.973994 3842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:00:00.974014 3842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:00:00.975205 3842 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:00:00.975246 3842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:00:00.975258 3842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:00:00.975271 3842 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:00:00.975283 3842 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:00:00.976539 3842 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:33117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:43313
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:43313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:00:00.976730 3842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:00:00.976972 3842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:00:00.982197 3848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:00:00.982228 3842 server_base.cc:1034] running on GCE node
W20260929 11:00:00.982389 3847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:00:00.982228 3850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:00:00.982683 3842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:00:00.983006 3842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:00:00.984169 3842 hybrid_clock.cc:648] HybridClock initialized: now 1790679600984153 us; error 50 us; skew 500 ppm
I20260929 11:00:00.985404 3842 webserver.cc:460] Webserver started at http://127.26.120.254:39809/ using document root <none> and password file <none>
I20260929 11:00:00.985646 3842 fs_manager.cc:359] Metadata directory not provided
I20260929 11:00:00.985715 3842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:00:00.985863 3842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:00:00.986950 3842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "74be5a0462df4e79934433362e9d2be1"
format_stamp: "Formatted at 2026-09-29 11:00:00 on dist-test-slave-2lzq"
server_key: "d2f71a9b2db7e498b8ab63868dea9206"
server_key_iv: "82dfe4e3a0f86130f5d16817b458ea30"
server_key_version: "encryptionkey@0"
I20260929 11:00:00.987305 3842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "74be5a0462df4e79934433362e9d2be1"
format_stamp: "Formatted at 2026-09-29 11:00:00 on dist-test-slave-2lzq"
server_key: "d2f71a9b2db7e498b8ab63868dea9206"
server_key_iv: "82dfe4e3a0f86130f5d16817b458ea30"
server_key_version: "encryptionkey@0"
I20260929 11:00:00.988598 3842 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260929 11:00:00.989264 3856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:00.989428 3842 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 11:00:00.989521 3842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "74be5a0462df4e79934433362e9d2be1"
format_stamp: "Formatted at 2026-09-29 11:00:00 on dist-test-slave-2lzq"
server_key: "d2f71a9b2db7e498b8ab63868dea9206"
server_key_iv: "82dfe4e3a0f86130f5d16817b458ea30"
server_key_version: "encryptionkey@0"
I20260929 11:00:00.989606 3842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:00:00.998245 3842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:00:00.998509 3842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:00:00.998656 3842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:00:01.002480 3842 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:43313
I20260929 11:00:01.002511 3908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:43313 every 8 connection(s)
I20260929 11:00:01.002866 3842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:00:01.003513 3909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:00:01.006107 3909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74be5a0462df4e79934433362e9d2be1: Bootstrap starting.
I20260929 11:00:01.006706 3909 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74be5a0462df4e79934433362e9d2be1: Neither blocks nor log segments found. Creating new log.
I20260929 11:00:01.006996 3909 log.cc:824] T 00000000000000000000000000000000 P 74be5a0462df4e79934433362e9d2be1: Log is configured to *not* fsync() on all Append() calls
I20260929 11:00:01.007665 3909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74be5a0462df4e79934433362e9d2be1: No bootstrap required, opened a new log
I20260929 11:00:01.008651 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 3842
I20260929 11:00:01.008752 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 11:00:01.008946 27107 external_mini_cluster.cc:1428] Setting key f8dd30b1079dceb2928149aca7c0b82c
I20260929 11:00:01.009480 3909 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74be5a0462df4e79934433362e9d2be1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74be5a0462df4e79934433362e9d2be1" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43313 } }
I20260929 11:00:01.009680 3909 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74be5a0462df4e79934433362e9d2be1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:00:01.009733 3909 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74be5a0462df4e79934433362e9d2be1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74be5a0462df4e79934433362e9d2be1, State: Initialized, Role: FOLLOWER
I20260929 11:00:01.009859 3909 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74be5a0462df4e79934433362e9d2be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74be5a0462df4e79934433362e9d2be1" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43313 } }
I20260929 11:00:01.009943 3909 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74be5a0462df4e79934433362e9d2be1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 11:00:01.009986 3909 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74be5a0462df4e79934433362e9d2be1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 11:00:01.010033 3909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74be5a0462df4e79934433362e9d2be1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:00:01.010586 3909 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74be5a0462df4e79934433362e9d2be1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74be5a0462df4e79934433362e9d2be1" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43313 } }
I20260929 11:00:01.010702 3909 leader_election.cc:302] T 00000000000000000000000000000000 P 74be5a0462df4e79934433362e9d2be1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74be5a0462df4e79934433362e9d2be1; no voters:
I20260929 11:00:01.010877 3909 leader_election.cc:288] T 00000000000000000000000000000000 P 74be5a0462df4e79934433362e9d2be1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 11:00:01.010934 3914 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74be5a0462df4e79934433362e9d2be1 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:00:01.011085 3914 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74be5a0462df4e79934433362e9d2be1 [term 1 LEADER]: Becoming Leader. State: Replica: 74be5a0462df4e79934433362e9d2be1, State: Running, Role: LEADER
I20260929 11:00:01.011138 3909 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74be5a0462df4e79934433362e9d2be1 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:00:01.011232 3914 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74be5a0462df4e79934433362e9d2be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74be5a0462df4e79934433362e9d2be1" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43313 } }
I20260929 11:00:01.012305 3915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74be5a0462df4e79934433362e9d2be1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74be5a0462df4e79934433362e9d2be1. Latest consensus state: current_term: 1 leader_uuid: "74be5a0462df4e79934433362e9d2be1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74be5a0462df4e79934433362e9d2be1" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43313 } } }
I20260929 11:00:01.012383 3915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74be5a0462df4e79934433362e9d2be1 [sys.catalog]: This master's current role is: LEADER
I20260929 11:00:01.012696 3916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74be5a0462df4e79934433362e9d2be1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74be5a0462df4e79934433362e9d2be1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74be5a0462df4e79934433362e9d2be1" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43313 } } }
W20260929 11:00:01.013039 3930 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 74be5a0462df4e79934433362e9d2be1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260929 11:00:01.013078 3916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74be5a0462df4e79934433362e9d2be1 [sys.catalog]: This master's current role is: LEADER
W20260929 11:00:01.013088 3930 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 11:00:01.013186 3928 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:00:01.013340 3928 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:00:01.015051 3928 catalog_manager.cc:1347] Generated new cluster ID: 30fd3c1ec97e4f4d8915a94eb99d7995
I20260929 11:00:01.015106 3928 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 11:00:01.015656 3913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:01.009480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54258 (local address 127.26.120.254:43313)
0929 11:00:01.009688 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0929 11:00:01.009696 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 11:00:01.009745 (+ 49us) server_negotiation.cc:407] Connection header received
0929 11:00:01.009858 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:00:01.009864 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:00:01.009930 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:00:01.010099 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:00:01.010342 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:01.012197 (+ 1855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:01.013322 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:01.013945 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:01.014094 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:01.014872 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:00:01.014876 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:00:01.014876 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:00:01.015084 (+ 208us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:00:01.015127 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:00:01.015142 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:00:01.015204 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 11:00:01.015387 (+ 183us) server_negotiation.cc:299] Negotiation successful
0929 11:00:01.015460 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":98,"threads_started":1}
I20260929 11:00:01.033274 3928 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 11:00:01.034031 3928 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:00:01.041791 3928 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74be5a0462df4e79934433362e9d2be1: Generated new TSK 0
I20260929 11:00:01.042014 3928 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 11:00:01.049173 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.193:0
--local_ip_for_outbound_sockets=127.26.120.193
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:43313
--builtin_ntp_servers=127.26.120.212:33117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260929 11:00:01.141037 3933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:00:01.141247 3933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:00:01.141268 3933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:00:01.142544 3933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:00:01.142587 3933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:00:01.142612 3933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.193
I20260929 11:00:01.143936 3933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:33117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:43313
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.193
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:00:01.144176 3933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:00:01.144506 3933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:00:01.149595 3938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:00:01.149740 3933 server_base.cc:1034] running on GCE node
W20260929 11:00:01.149607 3941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:00:01.149609 3939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:00:01.150125 3933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:00:01.150383 3933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:00:01.151551 3933 hybrid_clock.cc:648] HybridClock initialized: now 1790679601151538 us; error 31 us; skew 500 ppm
I20260929 11:00:01.152835 3933 webserver.cc:460] Webserver started at http://127.26.120.193:34901/ using document root <none> and password file <none>
I20260929 11:00:01.153074 3933 fs_manager.cc:359] Metadata directory not provided
I20260929 11:00:01.153143 3933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:00:01.153280 3933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:00:01.154445 3933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/data/instance:
uuid: "13fc53e3ea1f4957bce695179ef53801"
format_stamp: "Formatted at 2026-09-29 11:00:01 on dist-test-slave-2lzq"
server_key: "8b1dfe69152c3c72f77b95ec819de036"
server_key_iv: "23990a4b33793eec161cf242c0a6f25e"
server_key_version: "encryptionkey@0"
I20260929 11:00:01.154850 3933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance:
uuid: "13fc53e3ea1f4957bce695179ef53801"
format_stamp: "Formatted at 2026-09-29 11:00:01 on dist-test-slave-2lzq"
server_key: "8b1dfe69152c3c72f77b95ec819de036"
server_key_iv: "23990a4b33793eec161cf242c0a6f25e"
server_key_version: "encryptionkey@0"
I20260929 11:00:01.156162 3933 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 11:00:01.156946 3947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:01.157140 3933 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:00:01.157238 3933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/wal
uuid: "13fc53e3ea1f4957bce695179ef53801"
format_stamp: "Formatted at 2026-09-29 11:00:01 on dist-test-slave-2lzq"
server_key: "8b1dfe69152c3c72f77b95ec819de036"
server_key_iv: "23990a4b33793eec161cf242c0a6f25e"
server_key_version: "encryptionkey@0"
I20260929 11:00:01.157333 3933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:00:01.165817 3933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:00:01.166144 3933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:00:01.166288 3933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:00:01.166509 3933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:00:01.166822 3933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:00:01.166875 3933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:01.166914 3933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:00:01.166940 3933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:01.172663 3933 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.193:33349
I20260929 11:00:01.172712 4060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.193:33349 every 8 connection(s)
I20260929 11:00:01.173007 3933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
I20260929 11:00:01.174829 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 3933
I20260929 11:00:01.174928 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance
I20260929 11:00:01.175109 27107 external_mini_cluster.cc:1428] Setting key a137d4433f061658dd51bfc6abb7ca1c
I20260929 11:00:01.175993 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.194:0
--local_ip_for_outbound_sockets=127.26.120.194
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:43313
--builtin_ntp_servers=127.26.120.212:33117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260929 11:00:01.178526 3913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:01.173670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:38197 (local address 127.26.120.254:43313)
0929 11:00:01.173732 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0929 11:00:01.173734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:00:01.173980 (+ 246us) server_negotiation.cc:407] Connection header received
0929 11:00:01.174573 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:00:01.174574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:00:01.174589 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:00:01.174648 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:00:01.175399 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:01.175905 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:01.176983 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:01.177296 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:01.177382 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:01.178251 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:00:01.178253 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:00:01.178253 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:00:01.178257 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:00:01.178273 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:00:01.178277 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:00:01.178329 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 11:00:01.178422 (+ 93us) server_negotiation.cc:299] Negotiation successful
0929 11:00:01.178460 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 11:00:01.178841 4063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:01.173666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43313 (local address 127.26.120.193:38197)
0929 11:00:01.173859 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0929 11:00:01.173891 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0929 11:00:01.174463 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:00:01.174733 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:00:01.174737 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:00:01.174753 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:00:01.175317 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:01.175323 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:01.176002 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:01.176004 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:01.176869 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:01.176871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:01.177401 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:01.177402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:01.177534 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:01.178095 (+ 561us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 11:00:01.178095 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:00:01.178133 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 11:00:01.178159 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:00:01.178341 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:00:01.178347 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:00:01.178350 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 11:00:01.178425 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 11:00:01.178550 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":78,"threads_started":1}
I20260929 11:00:01.179352 4061 heartbeater.cc:347] Connected to a master server at 127.26.120.254:43313
I20260929 11:00:01.179474 4061 heartbeater.cc:464] Registering TS with master...
I20260929 11:00:01.179652 4061 heartbeater.cc:511] Master 127.26.120.254:43313 requested a full tablet report, sending...
I20260929 11:00:01.180195 3873 ts_manager.cc:194] Registered new tserver with Master: 13fc53e3ea1f4957bce695179ef53801 (127.26.120.193:33349)
I20260929 11:00:01.180799 3873 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.120.193:38197
W20260929 11:00:01.255559 4064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:00:01.255791 4064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:00:01.255823 4064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:00:01.256929 4064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:00:01.256992 4064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:00:01.257031 4064 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.194
I20260929 11:00:01.258363 4064 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:33117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:43313
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.194
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:00:01.258615 4064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:00:01.258894 4064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:00:01.264209 4070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:00:01.264209 4072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:00:01.264215 4069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:00:01.264374 4064 server_base.cc:1034] running on GCE node
I20260929 11:00:01.264571 4064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:00:01.264758 4064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:00:01.265893 4064 hybrid_clock.cc:648] HybridClock initialized: now 1790679601265882 us; error 27 us; skew 500 ppm
I20260929 11:00:01.267144 4064 webserver.cc:460] Webserver started at http://127.26.120.194:46367/ using document root <none> and password file <none>
I20260929 11:00:01.267373 4064 fs_manager.cc:359] Metadata directory not provided
I20260929 11:00:01.267418 4064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:00:01.267560 4064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:00:01.268613 4064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/data/instance:
uuid: "cabc7d9596594324935711f6edc5d9b3"
format_stamp: "Formatted at 2026-09-29 11:00:01 on dist-test-slave-2lzq"
server_key: "fa84cb7b33b57fe099f5e3a08b43b5b2"
server_key_iv: "c63f064e6c900882058a6cb7bd29c494"
server_key_version: "encryptionkey@0"
I20260929 11:00:01.268956 4064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance:
uuid: "cabc7d9596594324935711f6edc5d9b3"
format_stamp: "Formatted at 2026-09-29 11:00:01 on dist-test-slave-2lzq"
server_key: "fa84cb7b33b57fe099f5e3a08b43b5b2"
server_key_iv: "c63f064e6c900882058a6cb7bd29c494"
server_key_version: "encryptionkey@0"
I20260929 11:00:01.270402 4064 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 11:00:01.271166 4078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:01.271346 4064 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 11:00:01.271417 4064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/wal
uuid: "cabc7d9596594324935711f6edc5d9b3"
format_stamp: "Formatted at 2026-09-29 11:00:01 on dist-test-slave-2lzq"
server_key: "fa84cb7b33b57fe099f5e3a08b43b5b2"
server_key_iv: "c63f064e6c900882058a6cb7bd29c494"
server_key_version: "encryptionkey@0"
I20260929 11:00:01.271529 4064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:00:01.305529 4064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:00:01.305853 4064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:00:01.305991 4064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:00:01.306263 4064 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:00:01.306608 4064 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:00:01.306641 4064 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:01.306692 4064 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:00:01.306725 4064 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:01.312646 4064 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.194:32933
I20260929 11:00:01.312708 4191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.194:32933 every 8 connection(s)
I20260929 11:00:01.313033 4064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
I20260929 11:00:01.318212 3913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:01.313929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:48587 (local address 127.26.120.254:43313)
0929 11:00:01.314009 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0929 11:00:01.314012 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:00:01.314298 (+ 286us) server_negotiation.cc:407] Connection header received
0929 11:00:01.314788 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:00:01.314789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:00:01.314800 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:00:01.314852 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:00:01.315509 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:01.315996 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:01.316932 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:01.317116 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:01.317171 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:01.317974 (+ 803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:00:01.317975 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:00:01.317975 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:00:01.317979 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:00:01.317993 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:00:01.317997 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:00:01.318036 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0929 11:00:01.318117 (+ 81us) server_negotiation.cc:299] Negotiation successful
0929 11:00:01.318151 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 11:00:01.318409 4194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:01.313903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43313 (local address 127.26.120.194:48587)
0929 11:00:01.314176 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0929 11:00:01.314211 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 11:00:01.314694 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:00:01.314928 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:00:01.314931 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:00:01.314940 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:00:01.315437 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:01.315441 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:01.316075 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:01.316076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:01.316854 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:01.316856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:01.317193 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:01.317194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:01.317316 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:01.317823 (+ 507us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 11:00:01.317824 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:00:01.317853 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 11:00:01.317876 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:00:01.318043 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:00:01.318045 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:00:01.318047 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 11:00:01.318118 (+ 71us) client_negotiation.cc:240] Negotiation successful
0929 11:00:01.318195 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":118,"threads_started":1}
I20260929 11:00:01.318928 4192 heartbeater.cc:347] Connected to a master server at 127.26.120.254:43313
I20260929 11:00:01.319036 4192 heartbeater.cc:464] Registering TS with master...
I20260929 11:00:01.319212 4192 heartbeater.cc:511] Master 127.26.120.254:43313 requested a full tablet report, sending...
I20260929 11:00:01.319608 3873 ts_manager.cc:194] Registered new tserver with Master: cabc7d9596594324935711f6edc5d9b3 (127.26.120.194:32933)
I20260929 11:00:01.319944 3873 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.120.194:48587
I20260929 11:00:01.321317 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 4064
I20260929 11:00:01.321389 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance
I20260929 11:00:01.321532 27107 external_mini_cluster.cc:1428] Setting key d0aee151199f55cab3dfc98aa1699f98
I20260929 11:00:01.322418 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.195:0
--local_ip_for_outbound_sockets=127.26.120.195
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:43313
--builtin_ntp_servers=127.26.120.212:33117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260929 11:00:01.402282 4195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:00:01.402474 4195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:00:01.402494 4195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:00:01.403561 4195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:00:01.403595 4195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:00:01.403620 4195 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.195
I20260929 11:00:01.404850 4195 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:33117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:43313
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.195
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:00:01.405028 4195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:00:01.405251 4195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:00:01.410208 4201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:00:01.410208 4200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:00:01.410290 4203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:00:01.410440 4195 server_base.cc:1034] running on GCE node
I20260929 11:00:01.410785 4195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:00:01.411028 4195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:00:01.412184 4195 hybrid_clock.cc:648] HybridClock initialized: now 1790679601412169 us; error 26 us; skew 500 ppm
I20260929 11:00:01.413457 4195 webserver.cc:460] Webserver started at http://127.26.120.195:40201/ using document root <none> and password file <none>
I20260929 11:00:01.413697 4195 fs_manager.cc:359] Metadata directory not provided
I20260929 11:00:01.413765 4195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:00:01.413905 4195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:00:01.415055 4195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/data/instance:
uuid: "2f9990fe14a14de2b48159c973e4bcb7"
format_stamp: "Formatted at 2026-09-29 11:00:01 on dist-test-slave-2lzq"
server_key: "ae33e2977a81f137f2e9c845f97dbc56"
server_key_iv: "089036b28f4505f174ab2e8d8b190b8a"
server_key_version: "encryptionkey@0"
I20260929 11:00:01.415418 4195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance:
uuid: "2f9990fe14a14de2b48159c973e4bcb7"
format_stamp: "Formatted at 2026-09-29 11:00:01 on dist-test-slave-2lzq"
server_key: "ae33e2977a81f137f2e9c845f97dbc56"
server_key_iv: "089036b28f4505f174ab2e8d8b190b8a"
server_key_version: "encryptionkey@0"
I20260929 11:00:01.416706 4195 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 11:00:01.417460 4209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:01.417639 4195 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 11:00:01.417730 4195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/wal
uuid: "2f9990fe14a14de2b48159c973e4bcb7"
format_stamp: "Formatted at 2026-09-29 11:00:01 on dist-test-slave-2lzq"
server_key: "ae33e2977a81f137f2e9c845f97dbc56"
server_key_iv: "089036b28f4505f174ab2e8d8b190b8a"
server_key_version: "encryptionkey@0"
I20260929 11:00:01.417816 4195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:00:01.429754 4195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:00:01.430039 4195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:00:01.430230 4195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:00:01.430449 4195 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:00:01.430770 4195 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:00:01.430823 4195 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:01.430864 4195 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:00:01.430892 4195 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:01.436596 4195 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.195:44747
I20260929 11:00:01.436641 4322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.195:44747 every 8 connection(s)
I20260929 11:00:01.436957 4195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
I20260929 11:00:01.437171 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 4195
I20260929 11:00:01.437268 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance
I20260929 11:00:01.437484 27107 external_mini_cluster.cc:1428] Setting key 8419c8bd50abdb1dd8c3e26fd357967c
I20260929 11:00:01.442229 3913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:01.437693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:35295 (local address 127.26.120.254:43313)
0929 11:00:01.437747 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0929 11:00:01.437750 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:00:01.438022 (+ 272us) server_negotiation.cc:407] Connection header received
0929 11:00:01.438659 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:00:01.438660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:00:01.438670 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:00:01.438711 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:00:01.439346 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:01.439867 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:01.440824 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:01.441023 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:01.441088 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:01.441916 (+ 828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:00:01.441917 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:00:01.441917 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:00:01.441922 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:00:01.441934 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:00:01.441939 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:00:01.441987 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 11:00:01.442132 (+ 145us) server_negotiation.cc:299] Negotiation successful
0929 11:00:01.442169 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 11:00:01.442415 4325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:01.437706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43313 (local address 127.26.120.195:35295)
0929 11:00:01.437908 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0929 11:00:01.437947 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 11:00:01.438557 (+ 610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:00:01.438774 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:00:01.438777 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:00:01.438785 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:00:01.439273 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:01.439277 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:01.439948 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:01.439949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:01.440728 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:01.440730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:01.441109 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:01.441110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:01.441222 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:01.441775 (+ 553us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 11:00:01.441776 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:00:01.441803 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 11:00:01.441826 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:00:01.441996 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:00:01.442000 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:00:01.442002 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 11:00:01.442110 (+ 108us) client_negotiation.cc:240] Negotiation successful
0929 11:00:01.442216 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":106,"threads_started":1}
I20260929 11:00:01.442847 4323 heartbeater.cc:347] Connected to a master server at 127.26.120.254:43313
I20260929 11:00:01.442934 4323 heartbeater.cc:464] Registering TS with master...
I20260929 11:00:01.443075 4323 heartbeater.cc:511] Master 127.26.120.254:43313 requested a full tablet report, sending...
I20260929 11:00:01.443490 3873 ts_manager.cc:194] Registered new tserver with Master: 2f9990fe14a14de2b48159c973e4bcb7 (127.26.120.195:44747)
I20260929 11:00:01.443884 3873 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.120.195:35295
I20260929 11:00:01.449488 27107 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 11:00:01.451367 3913 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 11:00:01.450865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54274 (local address 127.26.120.254:43313)
0929 11:00:01.450924 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0929 11:00:01.450926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:00:01.451050 (+ 124us) server_negotiation.cc:407] Connection header received
0929 11:00:01.451089 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:00:01.451090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:00:01.451100 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:00:01.451151 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:00:01.451293 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54274: BlockingRecv error: recv got EOF from 127.0.0.1:54274 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 11:00:01.451442 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 3933
I20260929 11:00:01.455935 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 4064
I20260929 11:00:01.460500 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 4195
I20260929 11:00:01.464704 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 3842
2026-09-29T11:00:01Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (597 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-29T11:00:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T11:00:01Z Disabled control of system clock
I20260929 11:00:01.491511 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:41087
--webserver_interface=127.26.120.254
--webserver_port=0
--builtin_ntp_servers=127.26.120.212:39873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:41087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 11:00:01.571842 4334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:00:01.572045 4334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:00:01.572067 4334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:00:01.573210 4334 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:00:01.573256 4334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:00:01.573269 4334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:00:01.573282 4334 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:00:01.573342 4334 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:00:01.574699 4334 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:39873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:41087
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:41087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:00:01.574962 4334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:00:01.575259 4334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:00:01.580282 4340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:00:01.580260 4334 server_base.cc:1034] running on GCE node
W20260929 11:00:01.580242 4339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:00:01.580243 4342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:00:01.580814 4334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:00:01.581125 4334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:00:01.582286 4334 hybrid_clock.cc:648] HybridClock initialized: now 1790679601582236 us; error 68 us; skew 500 ppm
I20260929 11:00:01.583528 4334 webserver.cc:460] Webserver started at http://127.26.120.254:44351/ using document root <none> and password file <none>
I20260929 11:00:01.583765 4334 fs_manager.cc:359] Metadata directory not provided
I20260929 11:00:01.583846 4334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:00:01.584002 4334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:00:01.585045 4334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "1fe8f06635944701a1bd652ce1b99a59"
format_stamp: "Formatted at 2026-09-29 11:00:01 on dist-test-slave-2lzq"
server_key: "4f298fc2c7bc08b7886fcffb579e736e"
server_key_iv: "691fe9d0bfad17c5676d27a42ff13793"
server_key_version: "encryptionkey@0"
I20260929 11:00:01.585414 4334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "1fe8f06635944701a1bd652ce1b99a59"
format_stamp: "Formatted at 2026-09-29 11:00:01 on dist-test-slave-2lzq"
server_key: "4f298fc2c7bc08b7886fcffb579e736e"
server_key_iv: "691fe9d0bfad17c5676d27a42ff13793"
server_key_version: "encryptionkey@0"
I20260929 11:00:01.586766 4334 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 11:00:01.587549 4348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:01.587740 4334 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 11:00:01.587850 4334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "1fe8f06635944701a1bd652ce1b99a59"
format_stamp: "Formatted at 2026-09-29 11:00:01 on dist-test-slave-2lzq"
server_key: "4f298fc2c7bc08b7886fcffb579e736e"
server_key_iv: "691fe9d0bfad17c5676d27a42ff13793"
server_key_version: "encryptionkey@0"
I20260929 11:00:01.587929 4334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:00:01.591207 4334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:00:01.591447 4334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:00:01.591588 4334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:00:01.595279 4334 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:41087
I20260929 11:00:01.595316 4400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:41087 every 8 connection(s)
I20260929 11:00:01.595655 4334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:00:01.596310 4401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:00:01.599082 4401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fe8f06635944701a1bd652ce1b99a59: Bootstrap starting.
I20260929 11:00:01.599654 4401 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1fe8f06635944701a1bd652ce1b99a59: Neither blocks nor log segments found. Creating new log.
I20260929 11:00:01.599952 4401 log.cc:824] T 00000000000000000000000000000000 P 1fe8f06635944701a1bd652ce1b99a59: Log is configured to *not* fsync() on all Append() calls
I20260929 11:00:01.600654 4401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fe8f06635944701a1bd652ce1b99a59: No bootstrap required, opened a new log
I20260929 11:00:01.602305 4401 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1fe8f06635944701a1bd652ce1b99a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fe8f06635944701a1bd652ce1b99a59" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 41087 } }
I20260929 11:00:01.602448 4401 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1fe8f06635944701a1bd652ce1b99a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:00:01.602496 4401 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1fe8f06635944701a1bd652ce1b99a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fe8f06635944701a1bd652ce1b99a59, State: Initialized, Role: FOLLOWER
I20260929 11:00:01.602648 4401 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1fe8f06635944701a1bd652ce1b99a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fe8f06635944701a1bd652ce1b99a59" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 41087 } }
I20260929 11:00:01.602707 4401 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1fe8f06635944701a1bd652ce1b99a59 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 11:00:01.602728 4401 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1fe8f06635944701a1bd652ce1b99a59 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 11:00:01.602795 4401 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1fe8f06635944701a1bd652ce1b99a59 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:00:01.603327 4401 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1fe8f06635944701a1bd652ce1b99a59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fe8f06635944701a1bd652ce1b99a59" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 41087 } }
I20260929 11:00:01.603446 4401 leader_election.cc:302] T 00000000000000000000000000000000 P 1fe8f06635944701a1bd652ce1b99a59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1fe8f06635944701a1bd652ce1b99a59; no voters:
I20260929 11:00:01.603632 4401 leader_election.cc:288] T 00000000000000000000000000000000 P 1fe8f06635944701a1bd652ce1b99a59 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 11:00:01.603711 4404 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1fe8f06635944701a1bd652ce1b99a59 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:00:01.603899 4404 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1fe8f06635944701a1bd652ce1b99a59 [term 1 LEADER]: Becoming Leader. State: Replica: 1fe8f06635944701a1bd652ce1b99a59, State: Running, Role: LEADER
I20260929 11:00:01.603940 4401 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1fe8f06635944701a1bd652ce1b99a59 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:00:01.603981 4404 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1fe8f06635944701a1bd652ce1b99a59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fe8f06635944701a1bd652ce1b99a59" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 41087 } }
I20260929 11:00:01.604350 4405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fe8f06635944701a1bd652ce1b99a59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1fe8f06635944701a1bd652ce1b99a59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fe8f06635944701a1bd652ce1b99a59" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 41087 } } }
I20260929 11:00:01.604435 4405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fe8f06635944701a1bd652ce1b99a59 [sys.catalog]: This master's current role is: LEADER
I20260929 11:00:01.604616 4406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fe8f06635944701a1bd652ce1b99a59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1fe8f06635944701a1bd652ce1b99a59. Latest consensus state: current_term: 1 leader_uuid: "1fe8f06635944701a1bd652ce1b99a59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fe8f06635944701a1bd652ce1b99a59" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 41087 } } }
I20260929 11:00:01.604677 4406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fe8f06635944701a1bd652ce1b99a59 [sys.catalog]: This master's current role is: LEADER
I20260929 11:00:01.604964 4411 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:00:01.605417 4411 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:00:01.605700 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 4334
I20260929 11:00:01.605791 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 11:00:01.605983 27107 external_mini_cluster.cc:1428] Setting key 6503a5e8ed96229da245e5d17db45944
I20260929 11:00:01.607460 4411 catalog_manager.cc:1347] Generated new cluster ID: 9db7886c0aea47388c7063360670eab8
I20260929 11:00:01.607515 4411 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 11:00:01.608883 4423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:01.606647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33682 (local address 127.26.120.254:41087)
0929 11:00:01.606871 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0929 11:00:01.606880 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 11:00:01.606913 (+ 33us) server_negotiation.cc:407] Connection header received
0929 11:00:01.607029 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:00:01.607036 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:00:01.607082 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:00:01.607733 (+ 651us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:00:01.608276 (+ 543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:00:01.608280 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:00:01.608280 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:00:01.608284 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:00:01.608318 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:00:01.608331 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:00:01.608604 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0929 11:00:01.608634 (+ 30us) server_negotiation.cc:299] Negotiation successful
0929 11:00:01.608694 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":92,"threads_started":1}
I20260929 11:00:01.617100 4411 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 11:00:01.617266 4411 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:00:01.624617 4411 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1fe8f06635944701a1bd652ce1b99a59: Generated new TSK 0
I20260929 11:00:01.624830 4411 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 11:00:01.629513 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.193:0
--local_ip_for_outbound_sockets=127.26.120.193
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:41087
--builtin_ntp_servers=127.26.120.212:39873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 11:00:01.711222 4425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:00:01.711463 4425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:00:01.711489 4425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:00:01.712746 4425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:00:01.712788 4425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:00:01.712815 4425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.193
I20260929 11:00:01.714305 4425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:39873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:41087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.193
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:00:01.714579 4425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:00:01.714892 4425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:00:01.720067 4431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:00:01.720088 4430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:00:01.720371 4425 server_base.cc:1034] running on GCE node
W20260929 11:00:01.720062 4433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:00:01.720700 4425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:00:01.720983 4425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:00:01.722148 4425 hybrid_clock.cc:648] HybridClock initialized: now 1790679601722117 us; error 37 us; skew 500 ppm
I20260929 11:00:01.723601 4425 webserver.cc:460] Webserver started at http://127.26.120.193:34717/ using document root <none> and password file <none>
I20260929 11:00:01.723898 4425 fs_manager.cc:359] Metadata directory not provided
I20260929 11:00:01.723951 4425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:00:01.724108 4425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:00:01.725306 4425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/data/instance:
uuid: "a983b3d7a836470bb0ec2aebc224a148"
format_stamp: "Formatted at 2026-09-29 11:00:01 on dist-test-slave-2lzq"
server_key: "dc86cefdbaaa4bfba39f48437c008ad3"
server_key_iv: "6f9fa8a95361b9f2a1fe02cc46f06c78"
server_key_version: "encryptionkey@0"
I20260929 11:00:01.725701 4425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance:
uuid: "a983b3d7a836470bb0ec2aebc224a148"
format_stamp: "Formatted at 2026-09-29 11:00:01 on dist-test-slave-2lzq"
server_key: "dc86cefdbaaa4bfba39f48437c008ad3"
server_key_iv: "6f9fa8a95361b9f2a1fe02cc46f06c78"
server_key_version: "encryptionkey@0"
I20260929 11:00:01.727231 4425 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 11:00:01.728068 4439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:01.728235 4425 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 11:00:01.728397 4425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/wal
uuid: "a983b3d7a836470bb0ec2aebc224a148"
format_stamp: "Formatted at 2026-09-29 11:00:01 on dist-test-slave-2lzq"
server_key: "dc86cefdbaaa4bfba39f48437c008ad3"
server_key_iv: "6f9fa8a95361b9f2a1fe02cc46f06c78"
server_key_version: "encryptionkey@0"
I20260929 11:00:01.728593 4425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:00:01.732438 4425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:00:01.732736 4425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:00:01.732889 4425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:00:01.733125 4425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:00:01.733546 4425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:00:01.733590 4425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:01.733614 4425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:00:01.733628 4425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:01.740061 4425 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.193:32967
I20260929 11:00:01.740108 4552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.193:32967 every 8 connection(s)
I20260929 11:00:01.740456 4425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
I20260929 11:00:01.743728 4423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:01.741222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:49017 (local address 127.26.120.254:41087)
0929 11:00:01.741283 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 11:00:01.741286 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:00:01.741589 (+ 303us) server_negotiation.cc:407] Connection header received
0929 11:00:01.742101 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:00:01.742103 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:00:01.742109 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:00:01.742158 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:00:01.743448 (+ 1290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:00:01.743449 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:00:01.743450 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:00:01.743454 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:00:01.743473 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:00:01.743478 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:00:01.743542 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0929 11:00:01.743657 (+ 115us) server_negotiation.cc:299] Negotiation successful
0929 11:00:01.743674 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260929 11:00:01.743911 4555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:01.741217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:41087 (local address 127.26.120.193:49017)
0929 11:00:01.741437 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0929 11:00:01.741474 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 11:00:01.741982 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:00:01.742231 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:00:01.742235 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:00:01.742243 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:00:01.743271 (+ 1028us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 11:00:01.743271 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:00:01.743302 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 11:00:01.743334 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:00:01.743553 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:00:01.743558 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:00:01.743561 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 11:00:01.743629 (+ 68us) client_negotiation.cc:240] Negotiation successful
0929 11:00:01.743683 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":97,"threads_started":1}
I20260929 11:00:01.744554 4553 heartbeater.cc:347] Connected to a master server at 127.26.120.254:41087
I20260929 11:00:01.744648 4553 heartbeater.cc:464] Registering TS with master...
I20260929 11:00:01.744673 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 4425
I20260929 11:00:01.744796 4553 heartbeater.cc:511] Master 127.26.120.254:41087 requested a full tablet report, sending...
I20260929 11:00:01.744792 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance
I20260929 11:00:01.745018 27107 external_mini_cluster.cc:1428] Setting key f6ace4d7908061d189b56269562aa0f9
I20260929 11:00:01.745375 4365 ts_manager.cc:194] Registered new tserver with Master: a983b3d7a836470bb0ec2aebc224a148 (127.26.120.193:32967)
I20260929 11:00:01.746186 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.194:0
--local_ip_for_outbound_sockets=127.26.120.194
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:41087
--builtin_ntp_servers=127.26.120.212:39873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 11:00:01.825201 4556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:00:01.825407 4556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:00:01.825428 4556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:00:01.826527 4556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:00:01.826560 4556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:00:01.826583 4556 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.194
I20260929 11:00:01.827811 4556 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:39873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:41087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.194
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:00:01.827980 4556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:00:01.828200 4556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:00:01.833108 4564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:00:01.833127 4561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:00:01.833287 4556 server_base.cc:1034] running on GCE node
W20260929 11:00:01.833163 4562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:00:01.833549 4556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:00:01.833746 4556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:00:01.834879 4556 hybrid_clock.cc:648] HybridClock initialized: now 1790679601834855 us; error 41 us; skew 500 ppm
I20260929 11:00:01.836129 4556 webserver.cc:460] Webserver started at http://127.26.120.194:44603/ using document root <none> and password file <none>
I20260929 11:00:01.836366 4556 fs_manager.cc:359] Metadata directory not provided
I20260929 11:00:01.836413 4556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:00:01.836566 4556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:00:01.837620 4556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/data/instance:
uuid: "20cef902daf84aaeb0c580b4bb0aeda6"
format_stamp: "Formatted at 2026-09-29 11:00:01 on dist-test-slave-2lzq"
server_key: "f1e2a3addfcb655a5d61e837a6833e30"
server_key_iv: "3cc135830ceabde0e25e3183a75e6075"
server_key_version: "encryptionkey@0"
I20260929 11:00:01.837972 4556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance:
uuid: "20cef902daf84aaeb0c580b4bb0aeda6"
format_stamp: "Formatted at 2026-09-29 11:00:01 on dist-test-slave-2lzq"
server_key: "f1e2a3addfcb655a5d61e837a6833e30"
server_key_iv: "3cc135830ceabde0e25e3183a75e6075"
server_key_version: "encryptionkey@0"
I20260929 11:00:01.839238 4556 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 11:00:01.839970 4570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:01.840158 4556 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 11:00:01.840255 4556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/wal
uuid: "20cef902daf84aaeb0c580b4bb0aeda6"
format_stamp: "Formatted at 2026-09-29 11:00:01 on dist-test-slave-2lzq"
server_key: "f1e2a3addfcb655a5d61e837a6833e30"
server_key_iv: "3cc135830ceabde0e25e3183a75e6075"
server_key_version: "encryptionkey@0"
I20260929 11:00:01.840339 4556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:00:01.843689 4556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:00:01.843957 4556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:00:01.844105 4556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:00:01.844308 4556 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:00:01.844609 4556 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:00:01.844663 4556 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:01.844703 4556 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:00:01.844730 4556 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:01.850539 4556 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.194:45853
I20260929 11:00:01.850587 4683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.194:45853 every 8 connection(s)
I20260929 11:00:01.850901 4556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
I20260929 11:00:01.853834 4423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:01.851636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:36195 (local address 127.26.120.254:41087)
0929 11:00:01.851709 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0929 11:00:01.851711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:00:01.851954 (+ 243us) server_negotiation.cc:407] Connection header received
0929 11:00:01.852385 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:00:01.852386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:00:01.852393 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:00:01.852447 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:00:01.853648 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:00:01.853649 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:00:01.853649 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:00:01.853652 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:00:01.853664 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:00:01.853668 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:00:01.853711 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0929 11:00:01.853769 (+ 58us) server_negotiation.cc:299] Negotiation successful
0929 11:00:01.853781 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 11:00:01.854045 4686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:01.851644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:41087 (local address 127.26.120.194:36195)
0929 11:00:01.851829 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0929 11:00:01.851863 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 11:00:01.852304 (+ 441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:00:01.852533 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:00:01.852537 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:00:01.852544 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:00:01.853516 (+ 972us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 11:00:01.853517 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:00:01.853545 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 11:00:01.853573 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:00:01.853710 (+ 137us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:00:01.853713 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:00:01.853715 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 11:00:01.853767 (+ 52us) client_negotiation.cc:240] Negotiation successful
0929 11:00:01.853813 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":89,"threads_started":1}
I20260929 11:00:01.854545 4684 heartbeater.cc:347] Connected to a master server at 127.26.120.254:41087
I20260929 11:00:01.854650 4684 heartbeater.cc:464] Registering TS with master...
I20260929 11:00:01.854800 4684 heartbeater.cc:511] Master 127.26.120.254:41087 requested a full tablet report, sending...
I20260929 11:00:01.855166 4364 ts_manager.cc:194] Registered new tserver with Master: 20cef902daf84aaeb0c580b4bb0aeda6 (127.26.120.194:45853)
I20260929 11:00:01.860919 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 4556
I20260929 11:00:01.861003 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance
I20260929 11:00:01.861141 27107 external_mini_cluster.cc:1428] Setting key dbc88987f5e14f70774bc21d8ca9141a
I20260929 11:00:01.862097 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.195:0
--local_ip_for_outbound_sockets=127.26.120.195
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:41087
--builtin_ntp_servers=127.26.120.212:39873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 11:00:01.943477 4687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:00:01.943670 4687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:00:01.943727 4687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:00:01.944927 4687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:00:01.944967 4687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:00:01.944998 4687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.195
I20260929 11:00:01.946442 4687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:39873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:41087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.195
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:00:01.946692 4687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:00:01.946982 4687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:00:01.951905 4693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:00:01.951905 4695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:00:01.951915 4692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:00:01.952126 4687 server_base.cc:1034] running on GCE node
I20260929 11:00:01.952325 4687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:00:01.952513 4687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:00:01.953655 4687 hybrid_clock.cc:648] HybridClock initialized: now 1790679601953627 us; error 42 us; skew 500 ppm
I20260929 11:00:01.954975 4687 webserver.cc:460] Webserver started at http://127.26.120.195:35201/ using document root <none> and password file <none>
I20260929 11:00:01.955212 4687 fs_manager.cc:359] Metadata directory not provided
I20260929 11:00:01.955256 4687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:00:01.955404 4687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:00:01.956547 4687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/data/instance:
uuid: "fd78b60ab6a7416296df0dc7d6b37dca"
format_stamp: "Formatted at 2026-09-29 11:00:01 on dist-test-slave-2lzq"
server_key: "d1c52472eb5024ccf6d98451b28bad8c"
server_key_iv: "5083ac22a3e87e2328187962132d8bf5"
server_key_version: "encryptionkey@0"
I20260929 11:00:01.956915 4687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance:
uuid: "fd78b60ab6a7416296df0dc7d6b37dca"
format_stamp: "Formatted at 2026-09-29 11:00:01 on dist-test-slave-2lzq"
server_key: "d1c52472eb5024ccf6d98451b28bad8c"
server_key_iv: "5083ac22a3e87e2328187962132d8bf5"
server_key_version: "encryptionkey@0"
I20260929 11:00:01.958297 4687 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 11:00:01.959071 4701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:01.959239 4687 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 11:00:01.959327 4687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/wal
uuid: "fd78b60ab6a7416296df0dc7d6b37dca"
format_stamp: "Formatted at 2026-09-29 11:00:01 on dist-test-slave-2lzq"
server_key: "d1c52472eb5024ccf6d98451b28bad8c"
server_key_iv: "5083ac22a3e87e2328187962132d8bf5"
server_key_version: "encryptionkey@0"
I20260929 11:00:01.959399 4687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:00:01.962760 4687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:00:01.963029 4687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:00:01.963164 4687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:00:01.963363 4687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:00:01.963657 4687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:00:01.963689 4687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:01.963730 4687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:00:01.963755 4687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:01.969385 4687 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.195:35977
I20260929 11:00:01.969422 4814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.195:35977 every 8 connection(s)
I20260929 11:00:01.969748 4687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
I20260929 11:00:01.972680 4423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:01.970498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:44393 (local address 127.26.120.254:41087)
0929 11:00:01.970579 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0929 11:00:01.970582 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:00:01.970825 (+ 243us) server_negotiation.cc:407] Connection header received
0929 11:00:01.971264 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:00:01.971265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:00:01.971270 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:00:01.971323 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:00:01.972488 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:00:01.972489 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:00:01.972489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:00:01.972493 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:00:01.972504 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:00:01.972508 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:00:01.972554 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0929 11:00:01.972615 (+ 61us) server_negotiation.cc:299] Negotiation successful
0929 11:00:01.972628 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 11:00:01.972849 4817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:01.970466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:41087 (local address 127.26.120.195:44393)
0929 11:00:01.970662 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0929 11:00:01.970695 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0929 11:00:01.971178 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:00:01.971375 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:00:01.971378 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:00:01.971386 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:00:01.972347 (+ 961us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 11:00:01.972347 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:00:01.972375 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 11:00:01.972403 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:00:01.972552 (+ 149us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:00:01.972554 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:00:01.972556 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 11:00:01.972607 (+ 51us) client_negotiation.cc:240] Negotiation successful
0929 11:00:01.972654 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":95,"threads_started":1}
I20260929 11:00:01.973313 4815 heartbeater.cc:347] Connected to a master server at 127.26.120.254:41087
I20260929 11:00:01.973414 4815 heartbeater.cc:464] Registering TS with master...
I20260929 11:00:01.973560 4815 heartbeater.cc:511] Master 127.26.120.254:41087 requested a full tablet report, sending...
I20260929 11:00:01.973946 4364 ts_manager.cc:194] Registered new tserver with Master: fd78b60ab6a7416296df0dc7d6b37dca (127.26.120.195:35977)
I20260929 11:00:01.976814 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 4687
I20260929 11:00:01.976886 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance
I20260929 11:00:01.977025 27107 external_mini_cluster.cc:1428] Setting key fbef0e58c17a0ee6dcf3ae7b98a187a6
I20260929 11:00:01.977818 27107 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 11:00:01.979507 4423 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 11:00:01.979082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33698 (local address 127.26.120.254:41087)
0929 11:00:01.979130 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 11:00:01.979131 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 11:00:01.979232 (+ 101us) server_negotiation.cc:407] Connection header received
0929 11:00:01.979270 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:00:01.979271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:00:01.979274 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:00:01.979323 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:00:01.979428 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33698: BlockingRecv error: recv got EOF from 127.0.0.1:33698 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
I20260929 11:00:01.979643 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 4425
I20260929 11:00:01.984545 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 4556
I20260929 11:00:01.989378 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 4687
I20260929 11:00:01.993896 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 4334
2026-09-29T11:00:01Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (529 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 11:00:02 dist-test-slave-2lzq krb5kdc[4825](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 11:00:02 dist-test-slave-2lzq krb5kdc[4825](info): set up 2 sockets
Sep 29 11:00:02 dist-test-slave-2lzq krb5kdc[4825](info): commencing operation
krb5kdc: starting...
W20260929 11:00:03.031189 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 11:00:03 dist-test-slave-2lzq krb5kdc[4825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T11:00:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T11:00:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.120.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
I20260929 11:00:03.153609 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:35695
--webserver_interface=127.26.120.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:41615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:35695
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:00:03.233392 4838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:00:03.233611 4838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:00:03.233634 4838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:00:03.234841 4838 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:00:03.234884 4838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:00:03.234898 4838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:00:03.234911 4838 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:00:03.234923 4838 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:00:03.236403 4838 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:41615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:35695
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:35695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:00:03.236665 4838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:00:03.236907 4838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:00:03.241897 4843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:00:03.242000 4838 server_base.cc:1034] running on GCE node
W20260929 11:00:03.241905 4846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:00:03.241897 4844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:00:03.242364 4838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:00:03.242703 4838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:00:03.243867 4838 hybrid_clock.cc:648] HybridClock initialized: now 1790679603243861 us; error 53 us; skew 500 ppm
Sep 29 11:00:03 dist-test-slave-2lzq krb5kdc[4825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679603, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:00:03.246515 4838 init.cc:377] Logged in from keytab as kudu/127.26.120.254@KRBTEST.COM (short username kudu)
I20260929 11:00:03.246943 4838 webserver.cc:460] Webserver started at http://127.26.120.254:40553/ using document root <none> and password file <none>
I20260929 11:00:03.247205 4838 fs_manager.cc:359] Metadata directory not provided
I20260929 11:00:03.247274 4838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:00:03.247421 4838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:00:03.248538 4838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "aa53c87262494dbb8bd0488edf0e0741"
format_stamp: "Formatted at 2026-09-29 11:00:03 on dist-test-slave-2lzq"
server_key: "4a7905e74f4ed37a70634abef0fb8af5"
server_key_iv: "cdbf206c74636a922c722410b0edbbfb"
server_key_version: "encryptionkey@0"
I20260929 11:00:03.248898 4838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "aa53c87262494dbb8bd0488edf0e0741"
format_stamp: "Formatted at 2026-09-29 11:00:03 on dist-test-slave-2lzq"
server_key: "4a7905e74f4ed37a70634abef0fb8af5"
server_key_iv: "cdbf206c74636a922c722410b0edbbfb"
server_key_version: "encryptionkey@0"
I20260929 11:00:03.250334 4838 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 11:00:03.251245 4853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:03.251624 4838 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:00:03.251765 4838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "aa53c87262494dbb8bd0488edf0e0741"
format_stamp: "Formatted at 2026-09-29 11:00:03 on dist-test-slave-2lzq"
server_key: "4a7905e74f4ed37a70634abef0fb8af5"
server_key_iv: "cdbf206c74636a922c722410b0edbbfb"
server_key_version: "encryptionkey@0"
I20260929 11:00:03.251863 4838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:00:03.262490 4838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:00:03.264479 4838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:00:03.264652 4838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:00:03.269122 4838 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:35695
I20260929 11:00:03.269174 4905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:35695 every 8 connection(s)
I20260929 11:00:03.269572 4838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:00:03.270677 4906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:00:03.272905 4906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa53c87262494dbb8bd0488edf0e0741: Bootstrap starting.
I20260929 11:00:03.273483 4906 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa53c87262494dbb8bd0488edf0e0741: Neither blocks nor log segments found. Creating new log.
I20260929 11:00:03.273772 4906 log.cc:824] T 00000000000000000000000000000000 P aa53c87262494dbb8bd0488edf0e0741: Log is configured to *not* fsync() on all Append() calls
I20260929 11:00:03.274575 4906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa53c87262494dbb8bd0488edf0e0741: No bootstrap required, opened a new log
I20260929 11:00:03.276165 4906 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa53c87262494dbb8bd0488edf0e0741 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa53c87262494dbb8bd0488edf0e0741" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 35695 } }
I20260929 11:00:03.276296 4906 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa53c87262494dbb8bd0488edf0e0741 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:00:03.276319 4906 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa53c87262494dbb8bd0488edf0e0741 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa53c87262494dbb8bd0488edf0e0741, State: Initialized, Role: FOLLOWER
I20260929 11:00:03.276467 4906 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa53c87262494dbb8bd0488edf0e0741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa53c87262494dbb8bd0488edf0e0741" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 35695 } }
I20260929 11:00:03.276546 4906 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa53c87262494dbb8bd0488edf0e0741 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 11:00:03.276569 4906 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa53c87262494dbb8bd0488edf0e0741 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 11:00:03.276636 4906 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa53c87262494dbb8bd0488edf0e0741 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:00:03.277406 4906 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa53c87262494dbb8bd0488edf0e0741 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa53c87262494dbb8bd0488edf0e0741" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 35695 } }
I20260929 11:00:03.277518 4906 leader_election.cc:302] T 00000000000000000000000000000000 P aa53c87262494dbb8bd0488edf0e0741 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa53c87262494dbb8bd0488edf0e0741; no voters:
I20260929 11:00:03.277699 4906 leader_election.cc:288] T 00000000000000000000000000000000 P aa53c87262494dbb8bd0488edf0e0741 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 11:00:03.277786 4909 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa53c87262494dbb8bd0488edf0e0741 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:00:03.277985 4906 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa53c87262494dbb8bd0488edf0e0741 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:00:03.277971 4909 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa53c87262494dbb8bd0488edf0e0741 [term 1 LEADER]: Becoming Leader. State: Replica: aa53c87262494dbb8bd0488edf0e0741, State: Running, Role: LEADER
I20260929 11:00:03.278167 4909 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa53c87262494dbb8bd0488edf0e0741 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa53c87262494dbb8bd0488edf0e0741" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 35695 } }
I20260929 11:00:03.278535 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 4838
I20260929 11:00:03.278630 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 11:00:03.278589 4910 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa53c87262494dbb8bd0488edf0e0741 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa53c87262494dbb8bd0488edf0e0741" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa53c87262494dbb8bd0488edf0e0741" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 35695 } } }
I20260929 11:00:03.278610 4911 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa53c87262494dbb8bd0488edf0e0741 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa53c87262494dbb8bd0488edf0e0741. Latest consensus state: current_term: 1 leader_uuid: "aa53c87262494dbb8bd0488edf0e0741" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa53c87262494dbb8bd0488edf0e0741" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 35695 } } }
I20260929 11:00:03.278666 4910 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa53c87262494dbb8bd0488edf0e0741 [sys.catalog]: This master's current role is: LEADER
I20260929 11:00:03.278697 4911 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa53c87262494dbb8bd0488edf0e0741 [sys.catalog]: This master's current role is: LEADER
I20260929 11:00:03.278831 27107 external_mini_cluster.cc:1428] Setting key 60532fcd6564f9505a496094dad1a0df
I20260929 11:00:03.278887 4918 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:00:03.279361 4918 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:00:03.281181 4918 catalog_manager.cc:1347] Generated new cluster ID: e16b021c98f2461b91df3fc0f74a1689
I20260929 11:00:03.281255 4918 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 11:00:03 dist-test-slave-2lzq krb5kdc[4825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:00:03.287267 4918 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 11:00:03.287920 4918 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:00:03.292179 4927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:03.279492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60672 (local address 127.26.120.254:35695)
0929 11:00:03.279673 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 11:00:03.279683 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 11:00:03.279845 (+ 162us) server_negotiation.cc:407] Connection header received
0929 11:00:03.280155 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:00:03.280163 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:00:03.280223 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:00:03.280380 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:00:03.281145 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.282587 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.283603 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.284255 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.284369 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:03.287251 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:00:03.287256 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:00:03.287256 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:00:03.287268 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:00:03.290457 (+ 3189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:00:03.290844 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:00:03.290847 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:00:03.290847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:00:03.290949 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:00:03.291117 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:00:03.291118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:00:03.291118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:00:03.291366 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:00:03.291518 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0929 11:00:03.291741 (+ 223us) server_negotiation.cc:299] Negotiation successful
0929 11:00:03.291933 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":56,"threads_started":1}
I20260929 11:00:03.297237 4918 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa53c87262494dbb8bd0488edf0e0741: Generated new TSK 0
I20260929 11:00:03.297421 4918 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.120.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab.
Entry for principal kudu/127.26.120.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab.
I20260929 11:00:03.324838 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.193:0
--local_ip_for_outbound_sockets=127.26.120.193
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:35695
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:41615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:00:03.407148 4932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:00:03.407357 4932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:00:03.407382 4932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:00:03.408455 4932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:00:03.408492 4932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:00:03.408517 4932 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.193
I20260929 11:00:03.409873 4932 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:41615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:35695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.193
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:00:03.410111 4932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:00:03.410312 4932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:00:03.415182 4937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:00:03.415190 4940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:00:03.415218 4938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:00:03.415221 4932 server_base.cc:1034] running on GCE node
I20260929 11:00:03.415560 4932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:00:03.415803 4932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:00:03.416963 4932 hybrid_clock.cc:648] HybridClock initialized: now 1790679603416931 us; error 43 us; skew 500 ppm
Sep 29 11:00:03 dist-test-slave-2lzq krb5kdc[4825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679603, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:00:03.419528 4932 init.cc:377] Logged in from keytab as kudu/127.26.120.193@KRBTEST.COM (short username kudu)
I20260929 11:00:03.419961 4932 webserver.cc:460] Webserver started at http://127.26.120.193:39089/ using document root <none> and password file <none>
I20260929 11:00:03.420178 4932 fs_manager.cc:359] Metadata directory not provided
I20260929 11:00:03.420241 4932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:00:03.420382 4932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:00:03.421419 4932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/data/instance:
uuid: "18da9193b01d425ea82fc847972609dc"
format_stamp: "Formatted at 2026-09-29 11:00:03 on dist-test-slave-2lzq"
server_key: "8dc233cbe51143c16a34e589b049a468"
server_key_iv: "b45f89bb3d9c74e1127ce317d8f0171a"
server_key_version: "encryptionkey@0"
I20260929 11:00:03.421772 4932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance:
uuid: "18da9193b01d425ea82fc847972609dc"
format_stamp: "Formatted at 2026-09-29 11:00:03 on dist-test-slave-2lzq"
server_key: "8dc233cbe51143c16a34e589b049a468"
server_key_iv: "b45f89bb3d9c74e1127ce317d8f0171a"
server_key_version: "encryptionkey@0"
I20260929 11:00:03.423136 4932 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 11:00:03.423907 4947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:03.424088 4932 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:00:03.424182 4932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/wal
uuid: "18da9193b01d425ea82fc847972609dc"
format_stamp: "Formatted at 2026-09-29 11:00:03 on dist-test-slave-2lzq"
server_key: "8dc233cbe51143c16a34e589b049a468"
server_key_iv: "b45f89bb3d9c74e1127ce317d8f0171a"
server_key_version: "encryptionkey@0"
I20260929 11:00:03.424263 4932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:00:03.435286 4932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:00:03.437249 4932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:00:03.437438 4932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:00:03.437680 4932 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:00:03.437999 4932 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:00:03.438054 4932 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:03.438166 4932 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:00:03.438198 4932 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:03.444037 4932 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.193:42313
I20260929 11:00:03.444075 5060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.193:42313 every 8 connection(s)
I20260929 11:00:03.444432 4932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
I20260929 11:00:03.449661 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 4932
I20260929 11:00:03.449779 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance
I20260929 11:00:03.449965 27107 external_mini_cluster.cc:1428] Setting key a7e819e1cf3b69eb401ecfa39a638e42
Sep 29 11:00:03 dist-test-slave-2lzq krb5kdc[4825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679603, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:00:03.454855 4927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:03.445313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:33489 (local address 127.26.120.254:35695)
0929 11:00:03.445405 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0929 11:00:03.445408 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:00:03.445517 (+ 109us) server_negotiation.cc:407] Connection header received
0929 11:00:03.446021 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:00:03.446022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:00:03.446033 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:00:03.446117 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:00:03.447060 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.447617 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.448579 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.448795 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.449011 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:03.451684 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:00:03.451686 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:00:03.451686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:00:03.451694 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:00:03.453228 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:00:03.453733 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:00:03.453734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:00:03.453734 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:00:03.453786 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:00:03.454109 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:00:03.454110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:00:03.454111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:00:03.454224 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:00:03.454278 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 11:00:03.454651 (+ 373us) server_negotiation.cc:299] Negotiation successful
0929 11:00:03.454731 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 11:00:03.455077 5063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:03.445233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:35695 (local address 127.26.120.193:33489)
0929 11:00:03.445391 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0929 11:00:03.445434 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 11:00:03.445908 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:00:03.446200 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:00:03.446203 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:00:03.446768 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:00:03.446960 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:03.446965 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.447698 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.447698 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:03.448493 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:03.448495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.449031 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.449031 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:03.449136 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:03.449676 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:00:03.449685 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:00:03.451585 (+ 1900us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:00:03.453316 (+ 1731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:00:03.453324 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:00:03.453326 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:00:03.453656 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:00:03.453854 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:00:03.453855 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:00:03.453855 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:00:03.454003 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:00:03.454298 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:00:03.454306 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:00:03.454482 (+ 176us) client_negotiation.cc:769] Sending connection context
0929 11:00:03.454625 (+ 143us) client_negotiation.cc:240] Negotiation successful
0929 11:00:03.454819 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":88,"threads_started":1}
I20260929 11:00:03.455750 5061 heartbeater.cc:347] Connected to a master server at 127.26.120.254:35695
I20260929 11:00:03.455847 5061 heartbeater.cc:464] Registering TS with master...
I20260929 11:00:03.456105 5061 heartbeater.cc:511] Master 127.26.120.254:35695 requested a full tablet report, sending...
I20260929 11:00:03.456672 4870 ts_manager.cc:194] Registered new tserver with Master: 18da9193b01d425ea82fc847972609dc (127.26.120.193:42313)
I20260929 11:00:03.457270 4870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.193@KRBTEST.COM'} at 127.26.120.193:33489
WARNING: no policy specified for kudu/127.26.120.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab.
Entry for principal kudu/127.26.120.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab.
I20260929 11:00:03.476476 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.194:0
--local_ip_for_outbound_sockets=127.26.120.194
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:35695
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:41615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:00:03.559036 5066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:00:03.559283 5066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:00:03.559324 5066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:00:03.560482 5066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:00:03.560554 5066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:00:03.560598 5066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.194
I20260929 11:00:03.562166 5066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:41615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:35695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.194
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:00:03.562450 5066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:00:03.562683 5066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:00:03.567837 5072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:00:03.567842 5071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:00:03.567955 5066 server_base.cc:1034] running on GCE node
W20260929 11:00:03.567842 5074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:00:03.568267 5066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:00:03.568529 5066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:00:03.569703 5066 hybrid_clock.cc:648] HybridClock initialized: now 1790679603569685 us; error 40 us; skew 500 ppm
Sep 29 11:00:03 dist-test-slave-2lzq krb5kdc[4825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679603, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:00:03.572329 5066 init.cc:377] Logged in from keytab as kudu/127.26.120.194@KRBTEST.COM (short username kudu)
I20260929 11:00:03.572721 5066 webserver.cc:460] Webserver started at http://127.26.120.194:39379/ using document root <none> and password file <none>
I20260929 11:00:03.572914 5066 fs_manager.cc:359] Metadata directory not provided
I20260929 11:00:03.572957 5066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:00:03.573047 5066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:00:03.574031 5066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/data/instance:
uuid: "a19ea3be7d67404c9e1ec750f10b4532"
format_stamp: "Formatted at 2026-09-29 11:00:03 on dist-test-slave-2lzq"
server_key: "ecb24a7d77601d3679204433bdde045c"
server_key_iv: "b98ae81894665a0ae5e618a75b9d8263"
server_key_version: "encryptionkey@0"
I20260929 11:00:03.574412 5066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance:
uuid: "a19ea3be7d67404c9e1ec750f10b4532"
format_stamp: "Formatted at 2026-09-29 11:00:03 on dist-test-slave-2lzq"
server_key: "ecb24a7d77601d3679204433bdde045c"
server_key_iv: "b98ae81894665a0ae5e618a75b9d8263"
server_key_version: "encryptionkey@0"
I20260929 11:00:03.575667 5066 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:00:03.576316 5081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:03.576483 5066 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:03.576570 5066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/wal
uuid: "a19ea3be7d67404c9e1ec750f10b4532"
format_stamp: "Formatted at 2026-09-29 11:00:03 on dist-test-slave-2lzq"
server_key: "ecb24a7d77601d3679204433bdde045c"
server_key_iv: "b98ae81894665a0ae5e618a75b9d8263"
server_key_version: "encryptionkey@0"
I20260929 11:00:03.576627 5066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:00:03.586829 5066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:00:03.589119 5066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:00:03.589231 5066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:00:03.589403 5066 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:00:03.589726 5066 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:00:03.589761 5066 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:03.589807 5066 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:00:03.589833 5066 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:03.596002 5066 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.194:34985
I20260929 11:00:03.596081 5194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.194:34985 every 8 connection(s)
I20260929 11:00:03.596393 5066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
I20260929 11:00:03.601691 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 5066
I20260929 11:00:03.601783 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance
I20260929 11:00:03.601975 27107 external_mini_cluster.cc:1428] Setting key c69860575d4a371c530a6e1997f42e76
Sep 29 11:00:03 dist-test-slave-2lzq krb5kdc[4825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679603, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:00:03.606478 4927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:03.597149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:59713 (local address 127.26.120.254:35695)
0929 11:00:03.597213 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0929 11:00:03.597215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:00:03.597458 (+ 243us) server_negotiation.cc:407] Connection header received
0929 11:00:03.597991 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:00:03.597993 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:00:03.598004 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:00:03.598075 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:00:03.599002 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.599500 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.600520 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.600720 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.600794 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:03.603242 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:00:03.603243 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:00:03.603244 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:00:03.603252 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:00:03.604994 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:00:03.605399 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:00:03.605401 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:00:03.605402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:00:03.605466 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:00:03.605710 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:00:03.605711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:00:03.605711 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:00:03.605869 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:00:03.605937 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 11:00:03.606250 (+ 313us) server_negotiation.cc:299] Negotiation successful
0929 11:00:03.606338 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 11:00:03.606490 5197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:03.597127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:35695 (local address 127.26.120.194:59713)
0929 11:00:03.597297 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0929 11:00:03.597367 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0929 11:00:03.597884 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:00:03.598140 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:00:03.598144 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:00:03.598712 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:00:03.598905 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:03.598909 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.599605 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.599606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:03.600417 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:03.600419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.600800 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.600801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:03.600933 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:03.601434 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:00:03.601442 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:00:03.603148 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:00:03.605093 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:00:03.605099 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:00:03.605101 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:00:03.605321 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:00:03.605537 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:00:03.605538 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:00:03.605538 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:00:03.605645 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:00:03.605932 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:00:03.605936 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:00:03.606101 (+ 165us) client_negotiation.cc:769] Sending connection context
0929 11:00:03.606209 (+ 108us) client_negotiation.cc:240] Negotiation successful
0929 11:00:03.606329 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":77,"threads_started":1}
I20260929 11:00:03.607014 5195 heartbeater.cc:347] Connected to a master server at 127.26.120.254:35695
I20260929 11:00:03.607122 5195 heartbeater.cc:464] Registering TS with master...
I20260929 11:00:03.607321 5195 heartbeater.cc:511] Master 127.26.120.254:35695 requested a full tablet report, sending...
I20260929 11:00:03.607681 4870 ts_manager.cc:194] Registered new tserver with Master: a19ea3be7d67404c9e1ec750f10b4532 (127.26.120.194:34985)
I20260929 11:00:03.608062 4870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.194@KRBTEST.COM'} at 127.26.120.194:59713
WARNING: no policy specified for kudu/127.26.120.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab.
Entry for principal kudu/127.26.120.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab.
I20260929 11:00:03.627092 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.195:0
--local_ip_for_outbound_sockets=127.26.120.195
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:35695
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:41615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:00:03.708045 5200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:00:03.708245 5200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:00:03.708268 5200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:00:03.709406 5200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:00:03.709453 5200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:00:03.709481 5200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.195
I20260929 11:00:03.710866 5200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:41615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:35695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.195
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:00:03.711057 5200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:00:03.711283 5200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 11:00:03.716246 5200 server_base.cc:1034] running on GCE node
W20260929 11:00:03.716235 5206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:00:03.716213 5208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:00:03.716195 5205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:00:03.716639 5200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:00:03.716886 5200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:00:03.718047 5200 hybrid_clock.cc:648] HybridClock initialized: now 1790679603718039 us; error 52 us; skew 500 ppm
Sep 29 11:00:03 dist-test-slave-2lzq krb5kdc[4825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679603, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:00:03.728394 5200 init.cc:377] Logged in from keytab as kudu/127.26.120.195@KRBTEST.COM (short username kudu)
I20260929 11:00:03.728861 5200 webserver.cc:460] Webserver started at http://127.26.120.195:46371/ using document root <none> and password file <none>
I20260929 11:00:03.729107 5200 fs_manager.cc:359] Metadata directory not provided
I20260929 11:00:03.729178 5200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:00:03.729326 5200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:00:03.730470 5200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/data/instance:
uuid: "a4d221d98f714672afda5c402d4f048e"
format_stamp: "Formatted at 2026-09-29 11:00:03 on dist-test-slave-2lzq"
server_key: "0e3215408e54f8da01bb1c5232236b1e"
server_key_iv: "3cbdb1c21868dcf91e8605d4e2528612"
server_key_version: "encryptionkey@0"
I20260929 11:00:03.730844 5200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance:
uuid: "a4d221d98f714672afda5c402d4f048e"
format_stamp: "Formatted at 2026-09-29 11:00:03 on dist-test-slave-2lzq"
server_key: "0e3215408e54f8da01bb1c5232236b1e"
server_key_iv: "3cbdb1c21868dcf91e8605d4e2528612"
server_key_version: "encryptionkey@0"
I20260929 11:00:03.732154 5200 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 11:00:03.732896 5215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:03.733093 5200 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 11:00:03.733196 5200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/wal
uuid: "a4d221d98f714672afda5c402d4f048e"
format_stamp: "Formatted at 2026-09-29 11:00:03 on dist-test-slave-2lzq"
server_key: "0e3215408e54f8da01bb1c5232236b1e"
server_key_iv: "3cbdb1c21868dcf91e8605d4e2528612"
server_key_version: "encryptionkey@0"
I20260929 11:00:03.733268 5200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:00:03.761238 5200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:00:03.763314 5200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:00:03.763480 5200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:00:03.763724 5200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:00:03.764160 5200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:00:03.764207 5200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:03.764248 5200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:00:03.764273 5200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:03.770462 5200 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.195:45433
I20260929 11:00:03.770517 5328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.195:45433 every 8 connection(s)
I20260929 11:00:03.770843 5200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
I20260929 11:00:03.772179 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 5200
I20260929 11:00:03.772285 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance
I20260929 11:00:03.772497 27107 external_mini_cluster.cc:1428] Setting key 24183f6aa47ed2f02b91367818094134
Sep 29 11:00:03 dist-test-slave-2lzq krb5kdc[4825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679603, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:00:03.780908 4927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:03.771560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:49275 (local address 127.26.120.254:35695)
0929 11:00:03.771650 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0929 11:00:03.771652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:00:03.771839 (+ 187us) server_negotiation.cc:407] Connection header received
0929 11:00:03.772390 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:00:03.772391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:00:03.772403 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:00:03.772448 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:00:03.773687 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.774277 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.775290 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.775490 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.775580 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:03.777654 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:00:03.777657 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:00:03.777657 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:00:03.777664 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:00:03.779230 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:00:03.779874 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:00:03.779875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:00:03.779876 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:00:03.779938 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:00:03.780180 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:00:03.780180 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:00:03.780181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:00:03.780287 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:00:03.780342 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 11:00:03.780742 (+ 400us) server_negotiation.cc:299] Negotiation successful
0929 11:00:03.780842 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 11:00:03.781006 5331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:03.771520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:35695 (local address 127.26.120.195:49275)
0929 11:00:03.771710 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0929 11:00:03.771756 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 11:00:03.772296 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:00:03.772548 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:00:03.772553 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:00:03.773364 (+ 811us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:00:03.773579 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:03.773584 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.774368 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.774369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:03.775185 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:03.775187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.775589 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.775590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:03.775743 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:03.776201 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:00:03.776209 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:00:03.777539 (+ 1330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:00:03.779323 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:00:03.779330 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:00:03.779332 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:00:03.779778 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:00:03.780005 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:00:03.780005 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:00:03.780006 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:00:03.780116 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:00:03.780381 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:00:03.780387 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:00:03.780516 (+ 129us) client_negotiation.cc:769] Sending connection context
0929 11:00:03.780673 (+ 157us) client_negotiation.cc:240] Negotiation successful
0929 11:00:03.780805 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":95,"threads_started":1}
I20260929 11:00:03.781426 5329 heartbeater.cc:347] Connected to a master server at 127.26.120.254:35695
I20260929 11:00:03.781523 5329 heartbeater.cc:464] Registering TS with master...
I20260929 11:00:03.781666 5329 heartbeater.cc:511] Master 127.26.120.254:35695 requested a full tablet report, sending...
I20260929 11:00:03.782032 4870 ts_manager.cc:194] Registered new tserver with Master: a4d221d98f714672afda5c402d4f048e (127.26.120.195:45433)
I20260929 11:00:03.782423 4870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.195@KRBTEST.COM'} at 127.26.120.195:49275
I20260929 11:00:03.784078 27107 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 11:00:03.791003 4927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:03.785421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60676 (local address 127.26.120.254:35695)
0929 11:00:03.785471 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 11:00:03.785473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:00:03.785547 (+ 74us) server_negotiation.cc:407] Connection header received
0929 11:00:03.785598 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:00:03.785598 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:00:03.785606 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:00:03.785651 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:00:03.786217 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.786675 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.787367 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.787552 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.787605 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:03.788707 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:00:03.788708 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:00:03.788709 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:00:03.788714 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:00:03.790131 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:00:03.790410 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:00:03.790411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:00:03.790411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:00:03.790459 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:00:03.790602 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:00:03.790602 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:00:03.790603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:00:03.790723 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:00:03.790789 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 11:00:03.790878 (+ 89us) server_negotiation.cc:299] Negotiation successful
0929 11:00:03.790944 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260929 11:00:03.792575 4870 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60676:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 11:00:03.794165 4870 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 11:00:03.800434 5339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:03.796140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:34985 (local address 127.0.0.1:55370)
0929 11:00:03.796668 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0929 11:00:03.796677 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 11:00:03.796715 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:00:03.797031 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:00:03.797034 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:00:03.797037 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:00:03.797121 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:03.797124 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.798162 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.798163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:03.799403 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:03.799406 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.800106 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.800107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:03.800193 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:03.800211 (+ 18us) client_negotiation.cc:769] Sending connection context
0929 11:00:03.800298 (+ 87us) client_negotiation.cc:240] Negotiation successful
0929 11:00:03.800351 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":65,"threads_started":1}
I20260929 11:00:03.800433 5342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:03.796554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55370 (local address 127.26.120.194:34985)
0929 11:00:03.796693 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0929 11:00:03.796696 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:00:03.796708 (+ 12us) server_negotiation.cc:407] Connection header received
0929 11:00:03.796785 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:00:03.796787 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:00:03.796799 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:00:03.796894 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:00:03.797212 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.798014 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.799490 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.800020 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.800108 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:03.800164 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 11:00:03.800315 (+ 151us) server_negotiation.cc:299] Negotiation successful
0929 11:00:03.800355 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":88,"threads_started":1}
I20260929 11:00:03.800768 5341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:03.796384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43252 (local address 127.26.120.195:45433)
0929 11:00:03.796538 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0929 11:00:03.796542 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 11:00:03.796869 (+ 327us) server_negotiation.cc:407] Connection header received
0929 11:00:03.796923 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:00:03.796925 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:00:03.796940 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:00:03.797012 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:00:03.797235 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.798384 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.799582 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.800310 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.800384 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:03.800439 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 11:00:03.800648 (+ 209us) server_negotiation.cc:299] Negotiation successful
0929 11:00:03.800692 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":77,"threads_started":1}
I20260929 11:00:03.800762 5337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:03.796136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:42313 (local address 127.0.0.1:45906)
0929 11:00:03.796455 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0929 11:00:03.796497 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 11:00:03.796565 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:00:03.797035 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:00:03.797036 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:00:03.797040 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:00:03.797136 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:03.797138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.798503 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.798504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:03.799624 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:03.799626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.800537 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.800538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:03.800591 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:03.800594 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 11:00:03.800640 (+ 46us) client_negotiation.cc:240] Negotiation successful
0929 11:00:03.800667 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":46,"thread_start_us":65,"threads_started":1}
I20260929 11:00:03.800983 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:03.796221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45906 (local address 127.26.120.193:42313)
0929 11:00:03.796765 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0929 11:00:03.796769 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 11:00:03.796779 (+ 10us) server_negotiation.cc:407] Connection header received
0929 11:00:03.796823 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:00:03.796825 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:00:03.796842 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:00:03.796932 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:00:03.797220 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.798386 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.799713 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.800403 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.800464 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:03.800508 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0929 11:00:03.800860 (+ 352us) server_negotiation.cc:299] Negotiation successful
0929 11:00:03.800902 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":121,"threads_started":1}
I20260929 11:00:03.800762 5338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:03.796147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:45433 (local address 127.0.0.1:43252)
0929 11:00:03.796799 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0929 11:00:03.796808 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 11:00:03.796851 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:00:03.797035 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:00:03.797036 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:00:03.797038 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:00:03.797136 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:03.797139 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.798511 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.798512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:03.799433 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:03.799435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.800477 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.800478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:03.800545 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:03.800561 (+ 16us) client_negotiation.cc:769] Sending connection context
0929 11:00:03.800618 (+ 57us) client_negotiation.cc:240] Negotiation successful
0929 11:00:03.800664 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"mutex_wait_us":50,"thread_start_us":49,"threads_started":1}
I20260929 11:00:03.801725 4995 tablet_service.cc:1467] Processing CreateTablet for tablet 9ba2dedcec824ae9b528a53b9fd791b9 (DEFAULT_TABLE table=test-table [id=c5fb008119c7491fbeecec1e44f2e7db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:00:03.801730 5263 tablet_service.cc:1467] Processing CreateTablet for tablet 9ba2dedcec824ae9b528a53b9fd791b9 (DEFAULT_TABLE table=test-table [id=c5fb008119c7491fbeecec1e44f2e7db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:00:03.801730 5129 tablet_service.cc:1467] Processing CreateTablet for tablet 9ba2dedcec824ae9b528a53b9fd791b9 (DEFAULT_TABLE table=test-table [id=c5fb008119c7491fbeecec1e44f2e7db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:00:03.802096 5263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ba2dedcec824ae9b528a53b9fd791b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:00:03.802101 5129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ba2dedcec824ae9b528a53b9fd791b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:00:03.802098 4995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ba2dedcec824ae9b528a53b9fd791b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:00:03.804666 5343 tablet_bootstrap.cc:492] T 9ba2dedcec824ae9b528a53b9fd791b9 P 18da9193b01d425ea82fc847972609dc: Bootstrap starting.
I20260929 11:00:03.805235 5344 tablet_bootstrap.cc:492] T 9ba2dedcec824ae9b528a53b9fd791b9 P a4d221d98f714672afda5c402d4f048e: Bootstrap starting.
I20260929 11:00:03.805404 5343 tablet_bootstrap.cc:654] T 9ba2dedcec824ae9b528a53b9fd791b9 P 18da9193b01d425ea82fc847972609dc: Neither blocks nor log segments found. Creating new log.
I20260929 11:00:03.805485 5345 tablet_bootstrap.cc:492] T 9ba2dedcec824ae9b528a53b9fd791b9 P a19ea3be7d67404c9e1ec750f10b4532: Bootstrap starting.
I20260929 11:00:03.805919 5344 tablet_bootstrap.cc:654] T 9ba2dedcec824ae9b528a53b9fd791b9 P a4d221d98f714672afda5c402d4f048e: Neither blocks nor log segments found. Creating new log.
I20260929 11:00:03.806027 5345 tablet_bootstrap.cc:654] T 9ba2dedcec824ae9b528a53b9fd791b9 P a19ea3be7d67404c9e1ec750f10b4532: Neither blocks nor log segments found. Creating new log.
I20260929 11:00:03.806108 5343 log.cc:824] T 9ba2dedcec824ae9b528a53b9fd791b9 P 18da9193b01d425ea82fc847972609dc: Log is configured to *not* fsync() on all Append() calls
I20260929 11:00:03.806519 5344 log.cc:824] T 9ba2dedcec824ae9b528a53b9fd791b9 P a4d221d98f714672afda5c402d4f048e: Log is configured to *not* fsync() on all Append() calls
I20260929 11:00:03.806617 5345 log.cc:824] T 9ba2dedcec824ae9b528a53b9fd791b9 P a19ea3be7d67404c9e1ec750f10b4532: Log is configured to *not* fsync() on all Append() calls
I20260929 11:00:03.806942 5343 tablet_bootstrap.cc:492] T 9ba2dedcec824ae9b528a53b9fd791b9 P 18da9193b01d425ea82fc847972609dc: No bootstrap required, opened a new log
I20260929 11:00:03.807031 5343 ts_tablet_manager.cc:1397] T 9ba2dedcec824ae9b528a53b9fd791b9 P 18da9193b01d425ea82fc847972609dc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 11:00:03.807329 5344 tablet_bootstrap.cc:492] T 9ba2dedcec824ae9b528a53b9fd791b9 P a4d221d98f714672afda5c402d4f048e: No bootstrap required, opened a new log
I20260929 11:00:03.807360 5345 tablet_bootstrap.cc:492] T 9ba2dedcec824ae9b528a53b9fd791b9 P a19ea3be7d67404c9e1ec750f10b4532: No bootstrap required, opened a new log
I20260929 11:00:03.807399 5344 ts_tablet_manager.cc:1397] T 9ba2dedcec824ae9b528a53b9fd791b9 P a4d221d98f714672afda5c402d4f048e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 11:00:03.807456 5345 ts_tablet_manager.cc:1397] T 9ba2dedcec824ae9b528a53b9fd791b9 P a19ea3be7d67404c9e1ec750f10b4532: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 11:00:03.808866 5345 raft_consensus.cc:348] T 9ba2dedcec824ae9b528a53b9fd791b9 P a19ea3be7d67404c9e1ec750f10b4532 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a19ea3be7d67404c9e1ec750f10b4532" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 34985 } } peers { permanent_uuid: "18da9193b01d425ea82fc847972609dc" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42313 } } peers { permanent_uuid: "a4d221d98f714672afda5c402d4f048e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45433 } }
I20260929 11:00:03.809026 5345 raft_consensus.cc:374] T 9ba2dedcec824ae9b528a53b9fd791b9 P a19ea3be7d67404c9e1ec750f10b4532 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:00:03.809070 5345 raft_consensus.cc:729] T 9ba2dedcec824ae9b528a53b9fd791b9 P a19ea3be7d67404c9e1ec750f10b4532 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a19ea3be7d67404c9e1ec750f10b4532, State: Initialized, Role: FOLLOWER
I20260929 11:00:03.809012 5343 raft_consensus.cc:348] T 9ba2dedcec824ae9b528a53b9fd791b9 P 18da9193b01d425ea82fc847972609dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a19ea3be7d67404c9e1ec750f10b4532" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 34985 } } peers { permanent_uuid: "18da9193b01d425ea82fc847972609dc" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42313 } } peers { permanent_uuid: "a4d221d98f714672afda5c402d4f048e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45433 } }
I20260929 11:00:03.809176 5343 raft_consensus.cc:374] T 9ba2dedcec824ae9b528a53b9fd791b9 P 18da9193b01d425ea82fc847972609dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:00:03.809224 5343 raft_consensus.cc:729] T 9ba2dedcec824ae9b528a53b9fd791b9 P 18da9193b01d425ea82fc847972609dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18da9193b01d425ea82fc847972609dc, State: Initialized, Role: FOLLOWER
I20260929 11:00:03.809216 5345 consensus_queue.cc:260] T 9ba2dedcec824ae9b528a53b9fd791b9 P a19ea3be7d67404c9e1ec750f10b4532 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a19ea3be7d67404c9e1ec750f10b4532" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 34985 } } peers { permanent_uuid: "18da9193b01d425ea82fc847972609dc" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42313 } } peers { permanent_uuid: "a4d221d98f714672afda5c402d4f048e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45433 } }
I20260929 11:00:03.809338 5343 consensus_queue.cc:260] T 9ba2dedcec824ae9b528a53b9fd791b9 P 18da9193b01d425ea82fc847972609dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a19ea3be7d67404c9e1ec750f10b4532" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 34985 } } peers { permanent_uuid: "18da9193b01d425ea82fc847972609dc" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42313 } } peers { permanent_uuid: "a4d221d98f714672afda5c402d4f048e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45433 } }
I20260929 11:00:03.809298 5344 raft_consensus.cc:348] T 9ba2dedcec824ae9b528a53b9fd791b9 P a4d221d98f714672afda5c402d4f048e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a19ea3be7d67404c9e1ec750f10b4532" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 34985 } } peers { permanent_uuid: "18da9193b01d425ea82fc847972609dc" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42313 } } peers { permanent_uuid: "a4d221d98f714672afda5c402d4f048e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45433 } }
I20260929 11:00:03.809448 5344 raft_consensus.cc:374] T 9ba2dedcec824ae9b528a53b9fd791b9 P a4d221d98f714672afda5c402d4f048e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:00:03.809497 5344 raft_consensus.cc:729] T 9ba2dedcec824ae9b528a53b9fd791b9 P a4d221d98f714672afda5c402d4f048e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4d221d98f714672afda5c402d4f048e, State: Initialized, Role: FOLLOWER
I20260929 11:00:03.809506 5345 ts_tablet_manager.cc:1428] T 9ba2dedcec824ae9b528a53b9fd791b9 P a19ea3be7d67404c9e1ec750f10b4532: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 11:00:03.809624 5195 heartbeater.cc:503] Master 127.26.120.254:35695 was elected leader, sending a full tablet report...
I20260929 11:00:03.809633 5343 ts_tablet_manager.cc:1428] T 9ba2dedcec824ae9b528a53b9fd791b9 P 18da9193b01d425ea82fc847972609dc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 11:00:03.809602 5344 consensus_queue.cc:260] T 9ba2dedcec824ae9b528a53b9fd791b9 P a4d221d98f714672afda5c402d4f048e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a19ea3be7d67404c9e1ec750f10b4532" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 34985 } } peers { permanent_uuid: "18da9193b01d425ea82fc847972609dc" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42313 } } peers { permanent_uuid: "a4d221d98f714672afda5c402d4f048e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45433 } }
I20260929 11:00:03.809777 5061 heartbeater.cc:503] Master 127.26.120.254:35695 was elected leader, sending a full tablet report...
I20260929 11:00:03.809844 5344 ts_tablet_manager.cc:1428] T 9ba2dedcec824ae9b528a53b9fd791b9 P a4d221d98f714672afda5c402d4f048e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 11:00:03.810005 5329 heartbeater.cc:503] Master 127.26.120.254:35695 was elected leader, sending a full tablet report...
I20260929 11:00:03.830441 5350 raft_consensus.cc:482] T 9ba2dedcec824ae9b528a53b9fd791b9 P 18da9193b01d425ea82fc847972609dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 11:00:03.830559 5350 raft_consensus.cc:504] T 9ba2dedcec824ae9b528a53b9fd791b9 P 18da9193b01d425ea82fc847972609dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a19ea3be7d67404c9e1ec750f10b4532" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 34985 } } peers { permanent_uuid: "18da9193b01d425ea82fc847972609dc" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42313 } } peers { permanent_uuid: "a4d221d98f714672afda5c402d4f048e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45433 } }
I20260929 11:00:03.830984 5350 leader_election.cc:288] T 9ba2dedcec824ae9b528a53b9fd791b9 P 18da9193b01d425ea82fc847972609dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a19ea3be7d67404c9e1ec750f10b4532 (127.26.120.194:34985), a4d221d98f714672afda5c402d4f048e (127.26.120.195:45433)
I20260929 11:00:03.834904 5342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:03.831198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:50387 (local address 127.26.120.194:34985)
0929 11:00:03.831247 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 11:00:03.831249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:00:03.831280 (+ 31us) server_negotiation.cc:407] Connection header received
0929 11:00:03.831385 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:00:03.831386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:00:03.831404 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:00:03.831469 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:00:03.831679 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.832434 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.833950 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.834546 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.834623 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:03.834631 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 11:00:03.834794 (+ 163us) server_negotiation.cc:299] Negotiation successful
0929 11:00:03.834832 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 11:00:03.834919 5341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:03.831198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:38227 (local address 127.26.120.195:45433)
0929 11:00:03.831262 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0929 11:00:03.831264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:00:03.831475 (+ 211us) server_negotiation.cc:407] Connection header received
0929 11:00:03.831588 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:00:03.831590 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:00:03.831601 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:00:03.831660 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:00:03.831818 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.832503 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.833950 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.834548 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.834623 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:03.834631 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 11:00:03.834794 (+ 163us) server_negotiation.cc:299] Negotiation successful
0929 11:00:03.834832 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 11:00:03.834919 5063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:03.831150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:34985 (local address 127.26.120.193:50387)
0929 11:00:03.831231 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0929 11:00:03.831243 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 11:00:03.831295 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:00:03.831509 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:00:03.831510 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:00:03.831512 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:00:03.831597 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:03.831599 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.832510 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.832510 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:03.833849 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:03.833852 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.834629 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.834630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:03.834714 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:03.834716 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 11:00:03.834776 (+ 60us) client_negotiation.cc:240] Negotiation successful
0929 11:00:03.834826 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":49}
I20260929 11:00:03.834931 5352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:03.831153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:45433 (local address 127.26.120.193:38227)
0929 11:00:03.831404 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0929 11:00:03.831412 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 11:00:03.831481 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:00:03.831692 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:00:03.831693 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:00:03.831695 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:00:03.831757 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:03.831758 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.832574 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.832574 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:03.833849 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:03.833852 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.834629 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.834630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:03.834711 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:03.834715 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 11:00:03.834776 (+ 61us) client_negotiation.cc:240] Negotiation successful
0929 11:00:03.834826 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":79,"thread_start_us":118,"threads_started":1}
I20260929 11:00:03.835268 5149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ba2dedcec824ae9b528a53b9fd791b9" candidate_uuid: "18da9193b01d425ea82fc847972609dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a19ea3be7d67404c9e1ec750f10b4532" is_pre_election: true
I20260929 11:00:03.835426 5149 raft_consensus.cc:2454] T 9ba2dedcec824ae9b528a53b9fd791b9 P a19ea3be7d67404c9e1ec750f10b4532 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18da9193b01d425ea82fc847972609dc in term 0.
I20260929 11:00:03.835503 5283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ba2dedcec824ae9b528a53b9fd791b9" candidate_uuid: "18da9193b01d425ea82fc847972609dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4d221d98f714672afda5c402d4f048e" is_pre_election: true
I20260929 11:00:03.835676 5283 raft_consensus.cc:2454] T 9ba2dedcec824ae9b528a53b9fd791b9 P a4d221d98f714672afda5c402d4f048e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18da9193b01d425ea82fc847972609dc in term 0.
I20260929 11:00:03.835690 4949 leader_election.cc:302] T 9ba2dedcec824ae9b528a53b9fd791b9 P 18da9193b01d425ea82fc847972609dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18da9193b01d425ea82fc847972609dc, a19ea3be7d67404c9e1ec750f10b4532; no voters:
I20260929 11:00:03.835803 5350 raft_consensus.cc:2781] T 9ba2dedcec824ae9b528a53b9fd791b9 P 18da9193b01d425ea82fc847972609dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 11:00:03.835875 5350 raft_consensus.cc:482] T 9ba2dedcec824ae9b528a53b9fd791b9 P 18da9193b01d425ea82fc847972609dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 11:00:03.835892 5350 raft_consensus.cc:3037] T 9ba2dedcec824ae9b528a53b9fd791b9 P 18da9193b01d425ea82fc847972609dc [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:00:03.836627 5350 raft_consensus.cc:504] T 9ba2dedcec824ae9b528a53b9fd791b9 P 18da9193b01d425ea82fc847972609dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a19ea3be7d67404c9e1ec750f10b4532" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 34985 } } peers { permanent_uuid: "18da9193b01d425ea82fc847972609dc" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42313 } } peers { permanent_uuid: "a4d221d98f714672afda5c402d4f048e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45433 } }
I20260929 11:00:03.836795 5350 leader_election.cc:288] T 9ba2dedcec824ae9b528a53b9fd791b9 P 18da9193b01d425ea82fc847972609dc [CANDIDATE]: Term 1 election: Requested vote from peers a19ea3be7d67404c9e1ec750f10b4532 (127.26.120.194:34985), a4d221d98f714672afda5c402d4f048e (127.26.120.195:45433)
I20260929 11:00:03.836923 5149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ba2dedcec824ae9b528a53b9fd791b9" candidate_uuid: "18da9193b01d425ea82fc847972609dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a19ea3be7d67404c9e1ec750f10b4532"
I20260929 11:00:03.837005 5149 raft_consensus.cc:3037] T 9ba2dedcec824ae9b528a53b9fd791b9 P a19ea3be7d67404c9e1ec750f10b4532 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:00:03.837025 5283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ba2dedcec824ae9b528a53b9fd791b9" candidate_uuid: "18da9193b01d425ea82fc847972609dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4d221d98f714672afda5c402d4f048e"
I20260929 11:00:03.837103 5283 raft_consensus.cc:3037] T 9ba2dedcec824ae9b528a53b9fd791b9 P a4d221d98f714672afda5c402d4f048e [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:00:03.837743 5149 raft_consensus.cc:2454] T 9ba2dedcec824ae9b528a53b9fd791b9 P a19ea3be7d67404c9e1ec750f10b4532 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18da9193b01d425ea82fc847972609dc in term 1.
I20260929 11:00:03.837909 5283 raft_consensus.cc:2454] T 9ba2dedcec824ae9b528a53b9fd791b9 P a4d221d98f714672afda5c402d4f048e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18da9193b01d425ea82fc847972609dc in term 1.
I20260929 11:00:03.837939 4949 leader_election.cc:302] T 9ba2dedcec824ae9b528a53b9fd791b9 P 18da9193b01d425ea82fc847972609dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18da9193b01d425ea82fc847972609dc, a19ea3be7d67404c9e1ec750f10b4532; no voters:
I20260929 11:00:03.838040 5350 raft_consensus.cc:2781] T 9ba2dedcec824ae9b528a53b9fd791b9 P 18da9193b01d425ea82fc847972609dc [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:00:03.838218 5350 raft_consensus.cc:686] T 9ba2dedcec824ae9b528a53b9fd791b9 P 18da9193b01d425ea82fc847972609dc [term 1 LEADER]: Becoming Leader. State: Replica: 18da9193b01d425ea82fc847972609dc, State: Running, Role: LEADER
I20260929 11:00:03.838315 5350 consensus_queue.cc:237] T 9ba2dedcec824ae9b528a53b9fd791b9 P 18da9193b01d425ea82fc847972609dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a19ea3be7d67404c9e1ec750f10b4532" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 34985 } } peers { permanent_uuid: "18da9193b01d425ea82fc847972609dc" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42313 } } peers { permanent_uuid: "a4d221d98f714672afda5c402d4f048e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45433 } }
I20260929 11:00:03.839161 4870 catalog_manager.cc:5495] T 9ba2dedcec824ae9b528a53b9fd791b9 P 18da9193b01d425ea82fc847972609dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 18da9193b01d425ea82fc847972609dc (127.26.120.193). New cstate: current_term: 1 leader_uuid: "18da9193b01d425ea82fc847972609dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a19ea3be7d67404c9e1ec750f10b4532" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 34985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18da9193b01d425ea82fc847972609dc" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4d221d98f714672afda5c402d4f048e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45433 } health_report { overall_health: UNKNOWN } } }
I20260929 11:00:03.845155 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:03.843073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45916 (local address 127.26.120.193:42313)
0929 11:00:03.843111 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0929 11:00:03.843113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:00:03.843126 (+ 13us) server_negotiation.cc:407] Connection header received
0929 11:00:03.843152 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:00:03.843153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:00:03.843168 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:00:03.843234 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 11:00:03.843337 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.843783 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.844479 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:03.844659 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:03.844713 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:03.844858 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 11:00:03.844962 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 11:00:03.845027 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 11:00:03.845073 (+ 46us) server_negotiation.cc:299] Negotiation successful
0929 11:00:03.845091 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 11:00:03.846781 5283 raft_consensus.cc:1261] T 9ba2dedcec824ae9b528a53b9fd791b9 P a4d221d98f714672afda5c402d4f048e [term 1 FOLLOWER]: Refusing update from remote peer 18da9193b01d425ea82fc847972609dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 11:00:03.846827 5149 raft_consensus.cc:1261] T 9ba2dedcec824ae9b528a53b9fd791b9 P a19ea3be7d67404c9e1ec750f10b4532 [term 1 FOLLOWER]: Refusing update from remote peer 18da9193b01d425ea82fc847972609dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260929 11:00:03.846917 5196 tablet.cc:2369] T 9ba2dedcec824ae9b528a53b9fd791b9 P a19ea3be7d67404c9e1ec750f10b4532: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 11:00:03.847164 5350 consensus_queue.cc:1035] T 9ba2dedcec824ae9b528a53b9fd791b9 P 18da9193b01d425ea82fc847972609dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4d221d98f714672afda5c402d4f048e" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 11:00:03.847289 5353 consensus_queue.cc:1035] T 9ba2dedcec824ae9b528a53b9fd791b9 P 18da9193b01d425ea82fc847972609dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "a19ea3be7d67404c9e1ec750f10b4532" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 34985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 11:00:03.852048 4870 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 11:00:03.852157 4870 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 11:00:03.852931 4870 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aa53c87262494dbb8bd0488edf0e0741: Sending DeleteTablet for 3 replicas of tablet 9ba2dedcec824ae9b528a53b9fd791b9
I20260929 11:00:03.853231 4995 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ba2dedcec824ae9b528a53b9fd791b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:00:03 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:45906
I20260929 11:00:03.853287 5129 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ba2dedcec824ae9b528a53b9fd791b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:00:03 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:55370
I20260929 11:00:03.853541 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 4932
I20260929 11:00:03.853552 5367 tablet_replica.cc:330] T 9ba2dedcec824ae9b528a53b9fd791b9 P a19ea3be7d67404c9e1ec750f10b4532: stopping tablet replica
I20260929 11:00:03.853606 5263 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ba2dedcec824ae9b528a53b9fd791b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:00:03 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:43252
I20260929 11:00:03.853775 5367 raft_consensus.cc:2229] T 9ba2dedcec824ae9b528a53b9fd791b9 P a19ea3be7d67404c9e1ec750f10b4532 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 11:00:03.853919 5367 raft_consensus.cc:2258] T 9ba2dedcec824ae9b528a53b9fd791b9 P a19ea3be7d67404c9e1ec750f10b4532 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 11:00:03.854259 5368 tablet_replica.cc:330] T 9ba2dedcec824ae9b528a53b9fd791b9 P a4d221d98f714672afda5c402d4f048e: stopping tablet replica
I20260929 11:00:03.854368 5368 raft_consensus.cc:2229] T 9ba2dedcec824ae9b528a53b9fd791b9 P a4d221d98f714672afda5c402d4f048e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 11:00:03.854468 5368 raft_consensus.cc:2258] T 9ba2dedcec824ae9b528a53b9fd791b9 P a4d221d98f714672afda5c402d4f048e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 11:00:03.854980 5367 ts_tablet_manager.cc:1905] T 9ba2dedcec824ae9b528a53b9fd791b9 P a19ea3be7d67404c9e1ec750f10b4532: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 11:00:03.855970 5368 ts_tablet_manager.cc:1905] T 9ba2dedcec824ae9b528a53b9fd791b9 P a4d221d98f714672afda5c402d4f048e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 11:00:03.856966 5367 ts_tablet_manager.cc:1918] T 9ba2dedcec824ae9b528a53b9fd791b9 P a19ea3be7d67404c9e1ec750f10b4532: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 11:00:03.857048 5367 log.cc:1196] T 9ba2dedcec824ae9b528a53b9fd791b9 P a19ea3be7d67404c9e1ec750f10b4532: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-1/wal/wals/9ba2dedcec824ae9b528a53b9fd791b9
I20260929 11:00:03.857342 5367 ts_tablet_manager.cc:1939] T 9ba2dedcec824ae9b528a53b9fd791b9 P a19ea3be7d67404c9e1ec750f10b4532: Deleting consensus metadata
I20260929 11:00:03.857762 4855 catalog_manager.cc:4841] TS a19ea3be7d67404c9e1ec750f10b4532 (127.26.120.194:34985): tablet 9ba2dedcec824ae9b528a53b9fd791b9 (table test-table [id=c5fb008119c7491fbeecec1e44f2e7db]) successfully deleted
I20260929 11:00:03.857789 5368 ts_tablet_manager.cc:1918] T 9ba2dedcec824ae9b528a53b9fd791b9 P a4d221d98f714672afda5c402d4f048e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 11:00:03.857920 5368 log.cc:1196] T 9ba2dedcec824ae9b528a53b9fd791b9 P a4d221d98f714672afda5c402d4f048e: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790679549986039-27107-0/minicluster-data/ts-2/wal/wals/9ba2dedcec824ae9b528a53b9fd791b9
I20260929 11:00:03.858228 5368 ts_tablet_manager.cc:1939] T 9ba2dedcec824ae9b528a53b9fd791b9 P a4d221d98f714672afda5c402d4f048e: Deleting consensus metadata
I20260929 11:00:03.858603 4856 catalog_manager.cc:4841] TS a4d221d98f714672afda5c402d4f048e (127.26.120.195:45433): tablet 9ba2dedcec824ae9b528a53b9fd791b9 (table test-table [id=c5fb008119c7491fbeecec1e44f2e7db]) successfully deleted
W20260929 11:00:03.859222 4854 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.120.193:42313 (error 108)
I20260929 11:00:03.859385 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 5066
I20260929 11:00:03.859995 5338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:03.859510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:42313 (local address 127.0.0.1:45920)
0929 11:00:03.859923 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0929 11:00:03.859956 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.193:42313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394}
W20260929 11:00:03.860457 4854 catalog_manager.cc:4584] TS 18da9193b01d425ea82fc847972609dc (127.26.120.193:42313): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9ba2dedcec824ae9b528a53b9fd791b9: Network error: Client connection negotiation failed: client connection to 127.26.120.193:42313: connect: Connection refused (error 111)
I20260929 11:00:03.864391 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 5200
I20260929 11:00:03.868803 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 4838
2026-09-29T11:00:03Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1875 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 11:00:03 dist-test-slave-2lzq krb5kdc[5372](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 11:00:03 dist-test-slave-2lzq krb5kdc[5372](info): set up 2 sockets
Sep 29 11:00:03 dist-test-slave-2lzq krb5kdc[5372](info): commencing operation
krb5kdc: starting...
W20260929 11:00:04.895247 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 11:00:04 dist-test-slave-2lzq krb5kdc[5372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T11:00:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T11:00:04Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.26.120.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.26.120.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.26.120.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/rangeradmin_127.26.120.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.26.120.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/rangeradmin_127.26.120.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.26.120.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.26.120.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.26.120.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/rangerlookup_127.26.120.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.26.120.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/rangerlookup_127.26.120.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.26.120.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.120.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.120.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/HTTP_127.26.120.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.26.120.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/HTTP_127.26.120.212@KRBTEST.COM.keytab.
I20260929 11:00:05.056921 27107 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskkJyhTP/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/postgres -l logfile start
2026-09-29 11:00:07.565 UTC [5402] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-29 11:00:07.565 UTC [5402] LOG: listening on IPv4 address "127.26.120.212", port 45119
2026-09-29 11:00:07.568 UTC [5402] LOG: listening on Unix socket "/tmp/.s.PGSQL.45119"
2026-09-29 11:00:07.606 UTC [5405] LOG: database system was shut down at 2026-09-29 11:00:06 UTC
2026-09-29 11:00:07.610 UTC [5402] LOG: database system is ready to accept connections
I20260929 11:00:08.619892 27107 mini_postgres.cc:96] Postgres bound to 45119
2026-09-29 11:00:08.626 UTC [5413] FATAL: database "slave" does not exist
127.26.120.212:45119 - accepting connections
I20260929 11:00:08.627018 27107 mini_ranger.cc:161] Starting Ranger...
I20260929 11:00:08.642457 27107 mini_ranger.cc:84] Created miniranger Postgres user
I20260929 11:00:09.137806 27107 mini_ranger.cc:87] Created ranger Postgres database
I20260929 11:00:09.137874 27107 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ranger-admin
2026-09-29 11:00:09,529 [I] DB FLAVOR :POSTGRES
2026-09-29 11:00:09,530 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 11:00:09,530 [I] Checking connection..
2026-09-29 11:00:09,530 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:45119/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-29 11:00:09,794 [I] Checking connection passed.
2026-09-29 11:00:09,794 [I] --------- Verifying version history table ---------
2026-09-29 11:00:09,794 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:45119/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 11:00:10,060 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 11:00:10,060 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:45119/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 11:00:10,303 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 11:00:10,303 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-29 11:00:10,303 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:45119/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-29 11:00:10.541 UTC [5477] WARNING: there is no transaction in progress
2026-09-29 11:00:10,548 [I] create_dbversion_catalog.sql file imported successfully
2026-09-29 11:00:10,548 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:45119/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 11:00:10,789 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-29 11:00:10,789 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 11:00:10,790 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:45119/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-29 11:00:11,046 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:45119/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-29 11:00:11,282 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:45119/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-2lzq.c.gcp-upstream.internal','N') ;"
2026-09-29 11:00:11,513 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:45119/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 11:00:11,751 [I] Table x_portal_user does not exist in database ranger
2026-09-29 11:00:11,752 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:45119/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 11:00:12,007 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-29 11:00:12,007 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:45119/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 11:00:12,232 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:45119/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 11:00:12,460 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-29 11:00:12,461 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:45119/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-29 11:00:13.448 UTC [5585] WARNING: there is no transaction in progress
2026-09-29 11:00:13.464 UTC [5585] WARNING: there is no transaction in progress
2026-09-29 11:00:13.481 UTC [5585] WARNING: there is no transaction in progress
2026-09-29 11:00:13.495 UTC [5585] WARNING: there is no transaction in progress
2026-09-29 11:00:13.511 UTC [5585] WARNING: there is no transaction in progress
2026-09-29 11:00:13.531 UTC [5585] WARNING: there is no transaction in progress
2026-09-29 11:00:13.596 UTC [5585] WARNING: there is no transaction in progress
2026-09-29 11:00:13.603 UTC [5585] WARNING: there is no transaction in progress
2026-09-29 11:00:13.617 UTC [5585] WARNING: there is no transaction in progress
2026-09-29 11:00:13.627 UTC [5585] WARNING: there is no transaction in progress
2026-09-29 11:00:13.637 UTC [5585] WARNING: there is no transaction in progress
2026-09-29 11:00:13.651 UTC [5585] WARNING: there is no transaction in progress
2026-09-29 11:00:13.661 UTC [5585] WARNING: there is no transaction in progress
2026-09-29 11:00:13.668 UTC [5585] WARNING: there is no transaction in progress
2026-09-29 11:00:13.674 UTC [5585] WARNING: there is no transaction in progress
2026-09-29 11:00:13.681 UTC [5585] WARNING: there is no transaction in progress
2026-09-29 11:00:14,314 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-29 11:00:14,314 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:45119/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-29 11:00:14,542 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-29 11:00:14,542 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:45119/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 11:00:14,785 [I] Table x_portal_user already exists in database 'ranger'
2026-09-29 11:00:14,785 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:45119/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 11:00:15,045 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-29 11:00:15,046 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:45119/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 11:00:15,288 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:45119/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 11:00:15,520 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:45119/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-2lzq.c.gcp-upstream.internal';"
2026-09-29 11:00:15,775 [I] CORE_DB_SCHEMA import status has been updated
2026-09-29 11:00:15,775 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:45119/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-29 11:00:16,026 [I] DB_PATCHES have already been applied
I20260929 11:00:16.031772 27107 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ranger-admin:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskkJyhTP/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260929 11:00:16.031865 27107 mini_ranger.cc:193] Using host: 127.26.120.212
I20260929 11:00:16.034976 27107 mini_ranger.cc:227] Ranger admin URL: http://127.26.120.212:46721
Sep 29, 2026 11:00:16 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 11:00:16 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 11:00:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ranger-admin/ews/webapp
Sep 29, 2026 11:00:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 29, 2026 11:00:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 29, 2026 11:00:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ranger-admin/ews/webapp].
Sep 29, 2026 11:00:16 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260929 11:00:17.138116 27107 mini_ranger.cc:160] Time spent starting Ranger: real 8.511s user 0.000s sys 0.009s
Sep 29, 2026 11:00:37 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 29, 2026 11:00:37 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 29, 2026 11:00:37 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 29, 2026 11:00:37 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 29, 2026 11:00:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 29, 2026 11:00:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 29, 2026 11:00:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 29, 2026 11:00:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 29, 2026 11:00:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 29, 2026 11:00:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 29, 2026 11:00:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 29, 2026 11:00:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 29, 2026 11:00:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 29, 2026 11:00:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 29, 2026 11:00:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 29, 2026 11:00:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 29, 2026 11:00:37 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 29, 2026 11:00:38 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260929 11:00:39.355399 27107 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.26.120.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.26.120.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.26.120.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/rangerkms_127.26.120.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.26.120.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/rangerkms_127.26.120.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.26.120.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.120.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.120.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/HTTP_127.26.120.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.26.120.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/HTTP_127.26.120.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 29 11:00:39 dist-test-slave-2lzq krb5kdc[5372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679639, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260929 11:00:39.445434 27107 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260929 11:00:39.455113 27107 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260929 11:00:40.273159 27107 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260929 11:00:40.273247 27107 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ranger-kms
I20260929 11:00:40.273267 27107 mini_ranger_kms.cc:227] Using postgres at 127.26.120.212:45119
2026-09-29 11:00:40,754 [I] DB FLAVOR :POSTGRES
2026-09-29 11:00:40,754 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 11:00:40,754 [I] Checking connection
2026-09-29 11:00:40,754 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:45119/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-29 11:00:40,989 [I] connection success
2026-09-29 11:00:40,990 [I] --------- Verifying Ranger DB tables ---------
2026-09-29 11:00:40,990 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-29 11:00:40,990 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:45119/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-29 11:00:41,232 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-29 11:00:41,232 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 11:00:41,233 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-29 11:00:41,233 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:45119/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-29 11:00:41,483 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260929 11:00:41.960343 27107 mini_ranger_kms.cc:310] Created kms service
I20260929 11:00:42.162771 27107 mini_ranger_kms.cc:326] Created kudu user
I20260929 11:00:42.207218 27107 mini_ranger_kms.cc:343] Created rangerkms user
Sep 29 11:00:42 dist-test-slave-2lzq krb5kdc[5372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679639, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.26.120.212@KRBTEST.COM
I20260929 11:00:42.605858 27107 mini_ranger_kms.cc:384] Added ranger policy
I20260929 11:00:42.605993 27107 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ranger-kms:/tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260929 11:00:42.606014 27107 mini_ranger_kms.cc:240] Using host: 127.26.120.212
I20260929 11:00:42.610018 27107 mini_ranger_kms.cc:276] Ranger KMS PID: 5765
I20260929 11:00:42.610170 27107 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.26.120.212:37417
Sep 29, 2026 11:00:43 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 11:00:43 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 11:00:43 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 29, 2026 11:00:43 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 29, 2026 11:00:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ranger-kms/ews/webapp].
Sep 29, 2026 11:00:44 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260929 11:00:44.875350 27107 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.430s user 0.014s sys 0.220s
I20260929 11:00:44.875466 27107 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260929 11:00:44.875558 27107 mini_ranger_kms.cc:401] 127.26.120.212:37417/kms/v1/keys
Sep 29 11:00:44 dist-test-slave-2lzq krb5kdc[5372](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790679644, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.120.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 29 11:00:45 dist-test-slave-2lzq krb5kdc[5372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679644, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.120.212@KRBTEST.COM for HTTP/127.26.120.212@KRBTEST.COM
Sep 29 11:00:45 dist-test-slave-2lzq krb5kdc[5372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679644, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.120.212@KRBTEST.COM for HTTP/127.26.120.212@KRBTEST.COM
Sep 29 11:00:47 dist-test-slave-2lzq krb5kdc[5372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.26.120.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
I20260929 11:00:47.784813 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:44973
--webserver_interface=127.26.120.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:39577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:44973
--ranger_config_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.120.212:37417/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:00:47.863914 5816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:00:47.864159 5816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:00:47.864194 5816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:00:47.865309 5816 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:00:47.865381 5816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:00:47.865404 5816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:00:47.865427 5816 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:00:47.865454 5816 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:00:47.867735 5816 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:39577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/master-0/wal
--ranger_kms_url=127.26.120.212:37417/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:44973
--ranger_config_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:44973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:00:47.868011 5816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:00:47.868229 5816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 11:00:47.873236 5816 server_base.cc:1034] running on GCE node
W20260929 11:00:47.873271 5822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:00:47.873294 5821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:00:47.873440 5824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:00:47.873683 5816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:00:47.873970 5816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:00:47.875131 5816 hybrid_clock.cc:648] HybridClock initialized: now 1790679647875142 us; error 53 us; skew 500 ppm
Sep 29 11:00:47 dist-test-slave-2lzq krb5kdc[5372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679647, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:00:47.877655 5816 init.cc:377] Logged in from keytab as kudu/127.26.120.254@KRBTEST.COM (short username kudu)
I20260929 11:00:47.878149 5816 webserver.cc:460] Webserver started at http://127.26.120.254:42597/ using document root <none> and password file <none>
I20260929 11:00:47.878361 5816 fs_manager.cc:359] Metadata directory not provided
I20260929 11:00:47.878401 5816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:00:47.878540 5816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 11:00:47 dist-test-slave-2lzq krb5kdc[5372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679647, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for HTTP/127.26.120.212@KRBTEST.COM
I20260929 11:00:47.941118 5816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "374b71e63e8b4fa188fb5c17daa8c964"
format_stamp: "Formatted at 2026-09-29 11:00:47 on dist-test-slave-2lzq"
server_key: "8a8cedf9beb2d440c786dd8619518e7c"
server_key_iv: "2f851115ead43ba40302c899b44fea9e"
server_key_version: "kuduclusterkey@0"
I20260929 11:00:47.941641 5816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "374b71e63e8b4fa188fb5c17daa8c964"
format_stamp: "Formatted at 2026-09-29 11:00:47 on dist-test-slave-2lzq"
server_key: "8a8cedf9beb2d440c786dd8619518e7c"
server_key_iv: "2f851115ead43ba40302c899b44fea9e"
server_key_version: "kuduclusterkey@0"
I20260929 11:00:47.943810 5816 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260929 11:00:47.958158 5831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:47.958426 5816 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 11:00:47.958524 5816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "374b71e63e8b4fa188fb5c17daa8c964"
format_stamp: "Formatted at 2026-09-29 11:00:47 on dist-test-slave-2lzq"
server_key: "8a8cedf9beb2d440c786dd8619518e7c"
server_key_iv: "2f851115ead43ba40302c899b44fea9e"
server_key_version: "kuduclusterkey@0"
I20260929 11:00:47.958629 5816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:00:47.967590 5816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:00:47.969913 5816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:00:47.970118 5816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:00:47.975049 5816 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:44973
I20260929 11:00:47.975425 5816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:00:47.975818 5883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:44973 every 8 connection(s)
I20260929 11:00:47.976181 5884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:00:47.978660 5884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 374b71e63e8b4fa188fb5c17daa8c964: Bootstrap starting.
I20260929 11:00:47.979238 5884 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 374b71e63e8b4fa188fb5c17daa8c964: Neither blocks nor log segments found. Creating new log.
I20260929 11:00:47.979530 5884 log.cc:824] T 00000000000000000000000000000000 P 374b71e63e8b4fa188fb5c17daa8c964: Log is configured to *not* fsync() on all Append() calls
I20260929 11:00:47.980212 5884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 374b71e63e8b4fa188fb5c17daa8c964: No bootstrap required, opened a new log
I20260929 11:00:47.980665 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 5816
I20260929 11:00:47.980772 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 11:00:47.980969 27107 external_mini_cluster.cc:1428] Setting key a0a6c7d39498fe6aedacf7ac337ba456
I20260929 11:00:47.982981 5884 raft_consensus.cc:348] T 00000000000000000000000000000000 P 374b71e63e8b4fa188fb5c17daa8c964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "374b71e63e8b4fa188fb5c17daa8c964" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44973 } }
I20260929 11:00:47.983222 5884 raft_consensus.cc:374] T 00000000000000000000000000000000 P 374b71e63e8b4fa188fb5c17daa8c964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:00:47.983282 5884 raft_consensus.cc:729] T 00000000000000000000000000000000 P 374b71e63e8b4fa188fb5c17daa8c964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 374b71e63e8b4fa188fb5c17daa8c964, State: Initialized, Role: FOLLOWER
I20260929 11:00:47.983438 5884 consensus_queue.cc:260] T 00000000000000000000000000000000 P 374b71e63e8b4fa188fb5c17daa8c964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "374b71e63e8b4fa188fb5c17daa8c964" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44973 } }
I20260929 11:00:47.983548 5884 raft_consensus.cc:388] T 00000000000000000000000000000000 P 374b71e63e8b4fa188fb5c17daa8c964 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 11:00:47.983603 5884 raft_consensus.cc:482] T 00000000000000000000000000000000 P 374b71e63e8b4fa188fb5c17daa8c964 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 11:00:47.983673 5884 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 374b71e63e8b4fa188fb5c17daa8c964 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:00:47.984391 5884 raft_consensus.cc:504] T 00000000000000000000000000000000 P 374b71e63e8b4fa188fb5c17daa8c964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "374b71e63e8b4fa188fb5c17daa8c964" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44973 } }
I20260929 11:00:47.984508 5884 leader_election.cc:302] T 00000000000000000000000000000000 P 374b71e63e8b4fa188fb5c17daa8c964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 374b71e63e8b4fa188fb5c17daa8c964; no voters:
I20260929 11:00:47.984696 5884 leader_election.cc:288] T 00000000000000000000000000000000 P 374b71e63e8b4fa188fb5c17daa8c964 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 11:00:47.984754 5889 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 374b71e63e8b4fa188fb5c17daa8c964 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:00:47.984939 5889 raft_consensus.cc:686] T 00000000000000000000000000000000 P 374b71e63e8b4fa188fb5c17daa8c964 [term 1 LEADER]: Becoming Leader. State: Replica: 374b71e63e8b4fa188fb5c17daa8c964, State: Running, Role: LEADER
I20260929 11:00:47.985051 5884 sys_catalog.cc:564] T 00000000000000000000000000000000 P 374b71e63e8b4fa188fb5c17daa8c964 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:00:47.985049 5889 consensus_queue.cc:237] T 00000000000000000000000000000000 P 374b71e63e8b4fa188fb5c17daa8c964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "374b71e63e8b4fa188fb5c17daa8c964" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44973 } }
I20260929 11:00:47.986496 5889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 374b71e63e8b4fa188fb5c17daa8c964 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "374b71e63e8b4fa188fb5c17daa8c964" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "374b71e63e8b4fa188fb5c17daa8c964" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44973 } } }
I20260929 11:00:47.986599 5889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 374b71e63e8b4fa188fb5c17daa8c964 [sys.catalog]: This master's current role is: LEADER
I20260929 11:00:47.986724 5889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 374b71e63e8b4fa188fb5c17daa8c964 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 374b71e63e8b4fa188fb5c17daa8c964. Latest consensus state: current_term: 1 leader_uuid: "374b71e63e8b4fa188fb5c17daa8c964" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "374b71e63e8b4fa188fb5c17daa8c964" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44973 } } }
I20260929 11:00:47.986721 5884 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260929 11:00:47.986809 5889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 374b71e63e8b4fa188fb5c17daa8c964 [sys.catalog]: This master's current role is: LEADER
I20260929 11:00:47.987252 5903 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 29 11:00:47 dist-test-slave-2lzq krb5kdc[5372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:00:47.994033 5903 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:00:47.996701 5888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:47.981513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55346 (local address 127.26.120.254:44973)
0929 11:00:47.981760 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0929 11:00:47.981770 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 11:00:47.981803 (+ 33us) server_negotiation.cc:407] Connection header received
0929 11:00:47.982923 (+ 1120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:00:47.982932 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:00:47.983006 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:00:47.983160 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:00:47.983897 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:47.985265 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:47.986376 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:47.987001 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:47.987114 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:47.990920 (+ 3806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:00:47.990974 (+ 54us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:00:47.990975 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:00:47.991094 (+ 119us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:00:47.994362 (+ 3268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:00:47.994808 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:00:47.994812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:00:47.994813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:00:47.994956 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:00:47.995222 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:00:47.995223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:00:47.995223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:00:47.995725 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:00:47.995894 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0929 11:00:47.996051 (+ 157us) server_negotiation.cc:299] Negotiation successful
0929 11:00:47.996359 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":85,"threads_started":1}
I20260929 11:00:47.998886 5903 catalog_manager.cc:1347] Generated new cluster ID: 656052c6560a4d41a23ed3b7e92e3601
I20260929 11:00:47.998937 5903 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 11:00:48.013193 5903 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 11:00:48.014034 5903 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:00:48.017809 5903 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 374b71e63e8b4fa188fb5c17daa8c964: Generated new TSK 0
I20260929 11:00:48.018100 5903 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.120.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab.
Entry for principal kudu/127.26.120.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab.
I20260929 11:00:48.722638 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.120.212:37417/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.193:0
--local_ip_for_outbound_sockets=127.26.120.193
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:44973
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:39577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:00:48.808943 5929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:00:48.809196 5929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:00:48.809239 5929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:00:48.810581 5929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:00:48.810649 5929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:00:48.810696 5929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.193
I20260929 11:00:48.812196 5929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:39577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.26.120.212:37417/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:44973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.193
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:00:48.812454 5929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:00:48.812682 5929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:00:48.817754 5935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:00:48.817852 5937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:00:48.817957 5934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:00:48.818758 5929 server_base.cc:1034] running on GCE node
I20260929 11:00:48.818966 5929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:00:48.819228 5929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:00:48.820432 5929 hybrid_clock.cc:648] HybridClock initialized: now 1790679648820394 us; error 44 us; skew 500 ppm
Sep 29 11:00:48 dist-test-slave-2lzq krb5kdc[5372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679648, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:00:48.823019 5929 init.cc:377] Logged in from keytab as kudu/127.26.120.193@KRBTEST.COM (short username kudu)
I20260929 11:00:48.823545 5929 webserver.cc:460] Webserver started at http://127.26.120.193:34623/ using document root <none> and password file <none>
I20260929 11:00:48.823798 5929 fs_manager.cc:359] Metadata directory not provided
I20260929 11:00:48.823843 5929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:00:48.823971 5929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 11:00:48 dist-test-slave-2lzq krb5kdc[5372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679648, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for HTTP/127.26.120.212@KRBTEST.COM
I20260929 11:00:48.838879 5929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-0/data/instance:
uuid: "e205023eaa8343bfaed54d39d0f1941a"
format_stamp: "Formatted at 2026-09-29 11:00:48 on dist-test-slave-2lzq"
server_key: "d9af2c96394baaa3083da80608b1d422"
server_key_iv: "d87bb9281f940ece08e3c31d9ca13f3e"
server_key_version: "kuduclusterkey@0"
I20260929 11:00:48.839396 5929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance:
uuid: "e205023eaa8343bfaed54d39d0f1941a"
format_stamp: "Formatted at 2026-09-29 11:00:48 on dist-test-slave-2lzq"
server_key: "d9af2c96394baaa3083da80608b1d422"
server_key_iv: "d87bb9281f940ece08e3c31d9ca13f3e"
server_key_version: "kuduclusterkey@0"
I20260929 11:00:48.843674 5929 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.000s
I20260929 11:00:48.857003 5944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:48.857254 5929 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:00:48.857389 5929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-0/wal
uuid: "e205023eaa8343bfaed54d39d0f1941a"
format_stamp: "Formatted at 2026-09-29 11:00:48 on dist-test-slave-2lzq"
server_key: "d9af2c96394baaa3083da80608b1d422"
server_key_iv: "d87bb9281f940ece08e3c31d9ca13f3e"
server_key_version: "kuduclusterkey@0"
I20260929 11:00:48.857499 5929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:00:48.870975 5929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:00:48.873414 5929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:00:48.873564 5929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:00:48.873803 5929 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:00:48.874186 5929 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:00:48.874238 5929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:48.874269 5929 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:00:48.874325 5929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:48.880910 5929 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.193:46329
I20260929 11:00:48.881387 5929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
I20260929 11:00:48.882386 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 5929
I20260929 11:00:48.882495 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance
I20260929 11:00:48.882680 27107 external_mini_cluster.cc:1428] Setting key f38506bc136180892217822c229bfe08
I20260929 11:00:48.883306 6059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.193:46329 every 8 connection(s)
WARNING: no policy specified for kudu/127.26.120.194@KRBTEST.COM; defaulting to no policy
Sep 29 11:00:48 dist-test-slave-2lzq krb5kdc[5372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679648, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.194@KRBTEST.COM" created.
I20260929 11:00:48.906936 6063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:48.888424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:47457 (local address 127.26.120.254:44973)
0929 11:00:48.890249 (+ 1825us) server_negotiation.cc:206] Beginning negotiation
0929 11:00:48.890253 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 11:00:48.890266 (+ 13us) server_negotiation.cc:407] Connection header received
0929 11:00:48.890295 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:00:48.890296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:00:48.890310 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:00:48.890394 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:00:48.891226 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:48.892056 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:48.893621 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:48.893951 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:48.894022 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:48.903471 (+ 9449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:00:48.903473 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:00:48.903474 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:00:48.903483 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:00:48.905235 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:00:48.905801 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:00:48.905802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:00:48.905803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:00:48.905871 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:00:48.906218 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:00:48.906220 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:00:48.906220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:00:48.906353 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:00:48.906447 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 11:00:48.906735 (+ 288us) server_negotiation.cc:299] Negotiation successful
0929 11:00:48.906822 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":113,"threads_started":1}
I20260929 11:00:48.907142 6062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:48.883281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44973 (local address 127.26.120.193:47457)
0929 11:00:48.883443 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0929 11:00:48.883487 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 11:00:48.884113 (+ 626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:00:48.890530 (+ 6417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:00:48.890536 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:00:48.890793 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:00:48.891102 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:48.891109 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:48.892180 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:48.892181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:48.893508 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:48.893512 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:48.894121 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:48.894122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:48.894264 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:48.899855 (+ 5591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:00:48.899867 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:00:48.903329 (+ 3462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:00:48.905358 (+ 2029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:00:48.905366 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:00:48.905369 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:00:48.905723 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:00:48.905963 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:00:48.905964 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:00:48.905965 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:00:48.906147 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:00:48.906426 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:00:48.906432 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:00:48.906596 (+ 164us) client_negotiation.cc:769] Sending connection context
0929 11:00:48.906715 (+ 119us) client_negotiation.cc:240] Negotiation successful
0929 11:00:48.906877 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":88,"threads_started":1}
I20260929 11:00:48.907871 6060 heartbeater.cc:347] Connected to a master server at 127.26.120.254:44973
I20260929 11:00:48.907981 6060 heartbeater.cc:464] Registering TS with master...
I20260929 11:00:48.908197 6060 heartbeater.cc:511] Master 127.26.120.254:44973 requested a full tablet report, sending...
I20260929 11:00:48.908818 5848 ts_manager.cc:194] Registered new tserver with Master: e205023eaa8343bfaed54d39d0f1941a (127.26.120.193:46329)
I20260929 11:00:48.909713 5848 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.193@KRBTEST.COM'} at 127.26.120.193:47457
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab.
Entry for principal kudu/127.26.120.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab.
I20260929 11:00:48.921731 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.120.212:37417/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.194:0
--local_ip_for_outbound_sockets=127.26.120.194
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:44973
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:39577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:00:49.056368 6066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:00:49.056653 6066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:00:49.056689 6066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:00:49.058501 6066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:00:49.058570 6066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:00:49.058619 6066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.194
I20260929 11:00:49.061319 6066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:39577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.26.120.212:37417/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:44973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.194
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:00:49.061743 6066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:00:49.062141 6066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:00:49.068568 6073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:00:49.068596 6076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:00:49.068586 6074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:00:49.069097 6066 server_base.cc:1034] running on GCE node
I20260929 11:00:49.069306 6066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:00:49.069591 6066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:00:49.070761 6066 hybrid_clock.cc:648] HybridClock initialized: now 1790679649070667 us; error 108 us; skew 500 ppm
Sep 29 11:00:49 dist-test-slave-2lzq krb5kdc[5372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679649, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:00:49.074249 6066 init.cc:377] Logged in from keytab as kudu/127.26.120.194@KRBTEST.COM (short username kudu)
I20260929 11:00:49.074779 6066 webserver.cc:460] Webserver started at http://127.26.120.194:41097/ using document root <none> and password file <none>
I20260929 11:00:49.075101 6066 fs_manager.cc:359] Metadata directory not provided
I20260929 11:00:49.075165 6066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:00:49.075335 6066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 11:00:49 dist-test-slave-2lzq krb5kdc[5372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679649, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for HTTP/127.26.120.212@KRBTEST.COM
I20260929 11:00:49.089381 6066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-1/data/instance:
uuid: "a1125bb0550545709141aacb9e0e4287"
format_stamp: "Formatted at 2026-09-29 11:00:49 on dist-test-slave-2lzq"
server_key: "370aa6c3c6c5ffd6e83cb63461fcceab"
server_key_iv: "0d48833d18c5c011ac608f81351d3921"
server_key_version: "kuduclusterkey@0"
I20260929 11:00:49.089918 6066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance:
uuid: "a1125bb0550545709141aacb9e0e4287"
format_stamp: "Formatted at 2026-09-29 11:00:49 on dist-test-slave-2lzq"
server_key: "370aa6c3c6c5ffd6e83cb63461fcceab"
server_key_iv: "0d48833d18c5c011ac608f81351d3921"
server_key_version: "kuduclusterkey@0"
I20260929 11:00:49.092010 6066 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
Sep 29 11:00:49 dist-test-slave-2lzq krb5kdc[5372](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790679649, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:00:49.114465 6083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:49.119596 6066 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.001s sys 0.000s
I20260929 11:00:49.119860 6066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-1/wal
uuid: "a1125bb0550545709141aacb9e0e4287"
format_stamp: "Formatted at 2026-09-29 11:00:49 on dist-test-slave-2lzq"
server_key: "370aa6c3c6c5ffd6e83cb63461fcceab"
server_key_iv: "0d48833d18c5c011ac608f81351d3921"
server_key_version: "kuduclusterkey@0"
I20260929 11:00:49.120086 6066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:00:49.144810 6066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:00:49.147960 6066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:00:49.148264 6066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:00:49.148579 6066 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:00:49.149011 6066 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:00:49.149102 6066 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:49.149175 6066 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:00:49.149233 6066 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:49.156229 6066 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.194:42157
I20260929 11:00:49.156649 6066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
I20260929 11:00:49.158486 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 6066
I20260929 11:00:49.158581 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance
I20260929 11:00:49.158776 27107 external_mini_cluster.cc:1428] Setting key 1d208ce9ecefd5fcc2169c1e4bd6e481
I20260929 11:00:49.161322 6196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.194:42157 every 8 connection(s)
Sep 29 11:00:49 dist-test-slave-2lzq krb5kdc[5372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679649, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:00:49.170120 6063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:49.159436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:42623 (local address 127.26.120.254:44973)
0929 11:00:49.159530 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0929 11:00:49.159532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:00:49.159660 (+ 128us) server_negotiation.cc:407] Connection header received
0929 11:00:49.160290 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:00:49.160291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:00:49.160303 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:00:49.160333 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:00:49.161190 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:49.162079 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:49.163648 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:49.163916 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:49.163962 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:49.166572 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:00:49.166573 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:00:49.166574 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:00:49.166582 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:00:49.168434 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:00:49.168936 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:00:49.168938 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:00:49.168938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:00:49.168997 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:00:49.169281 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:00:49.169282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:00:49.169282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:00:49.169445 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:00:49.169489 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0929 11:00:49.169887 (+ 398us) server_negotiation.cc:299] Negotiation successful
0929 11:00:49.169980 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 11:00:49.170656 6199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:49.159218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44973 (local address 127.26.120.194:42623)
0929 11:00:49.159531 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0929 11:00:49.159577 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 11:00:49.160181 (+ 604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:00:49.160522 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:00:49.160526 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:00:49.160766 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:00:49.161088 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:49.161094 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:49.162265 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:49.162266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:49.163548 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:49.163551 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:49.164060 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:49.164061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:49.164207 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:49.164805 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:00:49.164816 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:00:49.166440 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:00:49.168548 (+ 2108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:00:49.168555 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:00:49.168557 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:00:49.168847 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:00:49.169069 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:00:49.169070 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:00:49.169070 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:00:49.169204 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:00:49.169552 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:00:49.169558 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:00:49.169726 (+ 168us) client_negotiation.cc:769] Sending connection context
0929 11:00:49.170231 (+ 505us) client_negotiation.cc:240] Negotiation successful
0929 11:00:49.170408 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":66,"threads_started":1}
WARNING: no policy specified for kudu/127.26.120.195@KRBTEST.COM; defaulting to no policy
I20260929 11:00:49.171212 6197 heartbeater.cc:347] Connected to a master server at 127.26.120.254:44973
I20260929 11:00:49.171304 6197 heartbeater.cc:464] Registering TS with master...
I20260929 11:00:49.171527 6197 heartbeater.cc:511] Master 127.26.120.254:44973 requested a full tablet report, sending...
I20260929 11:00:49.171903 5848 ts_manager.cc:194] Registered new tserver with Master: a1125bb0550545709141aacb9e0e4287 (127.26.120.194:42157)
I20260929 11:00:49.172400 5848 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.194@KRBTEST.COM'} at 127.26.120.194:42623
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab.
Entry for principal kudu/127.26.120.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab.
I20260929 11:00:49.195369 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.120.212:37417/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.195:0
--local_ip_for_outbound_sockets=127.26.120.195
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:44973
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:39577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:00:49.335120 6203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:00:49.335415 6203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:00:49.335457 6203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:00:49.337141 6203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:00:49.337239 6203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:00:49.337320 6203 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.195
I20260929 11:00:49.339550 6203 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:39577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.26.120.212:37417/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:44973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.195
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:00:49.339807 6203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:00:49.340145 6203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:00:49.346473 6208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:00:49.346487 6211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:00:49.346778 6209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:00:49.347012 6203 server_base.cc:1034] running on GCE node
I20260929 11:00:49.347229 6203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:00:49.347517 6203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:00:49.348717 6203 hybrid_clock.cc:648] HybridClock initialized: now 1790679649348695 us; error 34 us; skew 500 ppm
Sep 29 11:00:49 dist-test-slave-2lzq krb5kdc[5372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679649, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:00:49.351749 6203 init.cc:377] Logged in from keytab as kudu/127.26.120.195@KRBTEST.COM (short username kudu)
I20260929 11:00:49.352198 6203 webserver.cc:460] Webserver started at http://127.26.120.195:35955/ using document root <none> and password file <none>
I20260929 11:00:49.352451 6203 fs_manager.cc:359] Metadata directory not provided
I20260929 11:00:49.352514 6203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:00:49.352682 6203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 11:00:49 dist-test-slave-2lzq krb5kdc[5372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679649, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for HTTP/127.26.120.212@KRBTEST.COM
I20260929 11:00:49.372967 6203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-2/data/instance:
uuid: "0d9663cfb1be41018b883676469e802d"
format_stamp: "Formatted at 2026-09-29 11:00:49 on dist-test-slave-2lzq"
server_key: "d3b27be76368b6bafb4ab65a87fb56c6"
server_key_iv: "ec88afbfcf488db179fd2b1ba5b4d555"
server_key_version: "kuduclusterkey@0"
I20260929 11:00:49.373648 6203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance:
uuid: "0d9663cfb1be41018b883676469e802d"
format_stamp: "Formatted at 2026-09-29 11:00:49 on dist-test-slave-2lzq"
server_key: "d3b27be76368b6bafb4ab65a87fb56c6"
server_key_iv: "ec88afbfcf488db179fd2b1ba5b4d555"
server_key_version: "kuduclusterkey@0"
I20260929 11:00:49.378491 6203 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260929 11:00:49.395190 6218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:49.395452 6203 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:00:49.395552 6203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-2/wal
uuid: "0d9663cfb1be41018b883676469e802d"
format_stamp: "Formatted at 2026-09-29 11:00:49 on dist-test-slave-2lzq"
server_key: "d3b27be76368b6bafb4ab65a87fb56c6"
server_key_iv: "ec88afbfcf488db179fd2b1ba5b4d555"
server_key_version: "kuduclusterkey@0"
I20260929 11:00:49.395648 6203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:00:49.411665 6203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:00:49.413784 6203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:00:49.413939 6203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:00:49.414186 6203 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:00:49.414577 6203 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:00:49.414625 6203 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:49.414657 6203 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:00:49.414686 6203 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:00:49.422399 6203 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.195:38871
I20260929 11:00:49.422434 6331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.195:38871 every 8 connection(s)
I20260929 11:00:49.422845 6203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
I20260929 11:00:49.428957 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 6203
I20260929 11:00:49.429091 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance
I20260929 11:00:49.429333 27107 external_mini_cluster.cc:1428] Setting key f99851cd49429c90d1609c70add17cec
Sep 29 11:00:49 dist-test-slave-2lzq krb5kdc[5372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679649, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:00:49.434036 6063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:49.423602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:51517 (local address 127.26.120.254:44973)
0929 11:00:49.423649 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 11:00:49.423652 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:00:49.423873 (+ 221us) server_negotiation.cc:407] Connection header received
0929 11:00:49.424345 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:00:49.424346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:00:49.424359 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:00:49.424445 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:00:49.425150 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:49.426114 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:49.427397 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:49.427771 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:49.427856 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:49.430753 (+ 2897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:00:49.430755 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:00:49.430755 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:00:49.430764 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:00:49.432585 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:00:49.433011 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:00:49.433012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:00:49.433013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:00:49.433089 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:00:49.433348 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:00:49.433349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:00:49.433350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:00:49.433509 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:00:49.433588 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 11:00:49.433836 (+ 248us) server_negotiation.cc:299] Negotiation successful
0929 11:00:49.433951 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 11:00:49.434156 6334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:49.423563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44973 (local address 127.26.120.195:51517)
0929 11:00:49.423736 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0929 11:00:49.423778 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 11:00:49.424269 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:00:49.424548 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:00:49.424552 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:00:49.424754 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:00:49.425055 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:49.425060 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:49.426211 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:49.426212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:49.427292 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:49.427295 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:49.427874 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:49.427875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:49.427981 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:49.428576 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:00:49.428584 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:00:49.430647 (+ 2063us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:00:49.432684 (+ 2037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:00:49.432691 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:00:49.432694 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:00:49.432931 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:00:49.433174 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:00:49.433174 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:00:49.433174 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:00:49.433280 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:00:49.433580 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:00:49.433584 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:00:49.433710 (+ 126us) client_negotiation.cc:769] Sending connection context
0929 11:00:49.433795 (+ 85us) client_negotiation.cc:240] Negotiation successful
0929 11:00:49.433918 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":96,"threads_started":1}
I20260929 11:00:49.440294 6332 heartbeater.cc:347] Connected to a master server at 127.26.120.254:44973
I20260929 11:00:49.440411 6332 heartbeater.cc:464] Registering TS with master...
I20260929 11:00:49.440625 6332 heartbeater.cc:511] Master 127.26.120.254:44973 requested a full tablet report, sending...
I20260929 11:00:49.441052 5847 ts_manager.cc:194] Registered new tserver with Master: 0d9663cfb1be41018b883676469e802d (127.26.120.195:38871)
I20260929 11:00:49.441455 5847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.195@KRBTEST.COM'} at 127.26.120.195:51517
I20260929 11:00:49.446491 27107 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 11:00:49.456435 6063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:49.448247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55358 (local address 127.26.120.254:44973)
0929 11:00:49.448290 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 11:00:49.448293 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:00:49.448485 (+ 192us) server_negotiation.cc:407] Connection header received
0929 11:00:49.448608 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:00:49.448609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:00:49.448621 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:00:49.448653 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:00:49.449463 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:49.450261 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:49.451413 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:49.451717 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:49.451763 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:49.453403 (+ 1640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:00:49.453405 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:00:49.453405 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:00:49.453413 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:00:49.455177 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:00:49.455580 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:00:49.455581 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:00:49.455582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:00:49.455643 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:00:49.455861 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:00:49.455862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:00:49.455862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:00:49.455994 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:00:49.456101 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0929 11:00:49.456254 (+ 153us) server_negotiation.cc:299] Negotiation successful
0929 11:00:49.456336 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 11:00:49.458484 5848 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55358:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 11:00:49.460826 5848 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 11:00:49.470132 6344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:49.465859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:46329 (local address 127.0.0.1:51834)
0929 11:00:49.466169 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0929 11:00:49.466216 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 11:00:49.466330 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:00:49.466648 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:00:49.466651 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:00:49.466653 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:00:49.466725 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:49.466728 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:49.467612 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:49.467612 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:49.468767 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:49.468770 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:49.469820 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:49.469821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:49.469901 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:49.469916 (+ 15us) client_negotiation.cc:769] Sending connection context
0929 11:00:49.469982 (+ 66us) client_negotiation.cc:240] Negotiation successful
0929 11:00:49.470032 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":87,"threads_started":1}
I20260929 11:00:49.470583 6345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:49.465938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51834 (local address 127.26.120.193:46329)
0929 11:00:49.466096 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0929 11:00:49.466102 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 11:00:49.466284 (+ 182us) server_negotiation.cc:407] Connection header received
0929 11:00:49.466457 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:00:49.466461 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:00:49.466478 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:00:49.466535 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:00:49.466811 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:49.467532 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:49.468869 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:49.469617 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:49.469680 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:49.469758 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 11:00:49.470471 (+ 713us) server_negotiation.cc:299] Negotiation successful
0929 11:00:49.470515 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":51,"threads_started":1}
I20260929 11:00:49.471554 5993 tablet_service.cc:1467] Processing CreateTablet for tablet 0e604636b0644f9381ce8f8d703517f5 (DEFAULT_TABLE table=test-table [id=316eafd8ce894109a70e5e810b424090]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:00:49.471917 5993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e604636b0644f9381ce8f8d703517f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:00:49.475255 6350 tablet_bootstrap.cc:492] T 0e604636b0644f9381ce8f8d703517f5 P e205023eaa8343bfaed54d39d0f1941a: Bootstrap starting.
I20260929 11:00:49.476009 6350 tablet_bootstrap.cc:654] T 0e604636b0644f9381ce8f8d703517f5 P e205023eaa8343bfaed54d39d0f1941a: Neither blocks nor log segments found. Creating new log.
I20260929 11:00:49.476372 6350 log.cc:824] T 0e604636b0644f9381ce8f8d703517f5 P e205023eaa8343bfaed54d39d0f1941a: Log is configured to *not* fsync() on all Append() calls
I20260929 11:00:49.477733 6350 tablet_bootstrap.cc:492] T 0e604636b0644f9381ce8f8d703517f5 P e205023eaa8343bfaed54d39d0f1941a: No bootstrap required, opened a new log
I20260929 11:00:49.477797 6350 ts_tablet_manager.cc:1397] T 0e604636b0644f9381ce8f8d703517f5 P e205023eaa8343bfaed54d39d0f1941a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 11:00:49.480438 6350 raft_consensus.cc:348] T 0e604636b0644f9381ce8f8d703517f5 P e205023eaa8343bfaed54d39d0f1941a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e205023eaa8343bfaed54d39d0f1941a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46329 } } peers { permanent_uuid: "0d9663cfb1be41018b883676469e802d" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 38871 } } peers { permanent_uuid: "a1125bb0550545709141aacb9e0e4287" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 42157 } }
I20260929 11:00:49.480598 6350 raft_consensus.cc:374] T 0e604636b0644f9381ce8f8d703517f5 P e205023eaa8343bfaed54d39d0f1941a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:00:49.480626 6350 raft_consensus.cc:729] T 0e604636b0644f9381ce8f8d703517f5 P e205023eaa8343bfaed54d39d0f1941a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e205023eaa8343bfaed54d39d0f1941a, State: Initialized, Role: FOLLOWER
I20260929 11:00:49.481341 6350 consensus_queue.cc:260] T 0e604636b0644f9381ce8f8d703517f5 P e205023eaa8343bfaed54d39d0f1941a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e205023eaa8343bfaed54d39d0f1941a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46329 } } peers { permanent_uuid: "0d9663cfb1be41018b883676469e802d" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 38871 } } peers { permanent_uuid: "a1125bb0550545709141aacb9e0e4287" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 42157 } }
I20260929 11:00:49.481710 6350 ts_tablet_manager.cc:1428] T 0e604636b0644f9381ce8f8d703517f5 P e205023eaa8343bfaed54d39d0f1941a: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260929 11:00:49.481858 6060 heartbeater.cc:503] Master 127.26.120.254:44973 was elected leader, sending a full tablet report...
I20260929 11:00:49.483851 6348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:49.466301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:42157 (local address 127.0.0.1:35176)
0929 11:00:49.475934 (+ 9633us) negotiation.cc:107] Waiting for socket to connect
0929 11:00:49.475943 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 11:00:49.475987 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:00:49.476413 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:00:49.476414 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:00:49.476416 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:00:49.476911 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:49.476913 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:49.479552 (+ 2639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:49.479553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:49.480696 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:49.480698 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:49.483401 (+ 2703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:49.483403 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:49.483600 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:49.483604 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 11:00:49.483726 (+ 122us) client_negotiation.cc:240] Negotiation successful
0929 11:00:49.483775 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9616,"thread_start_us":59,"threads_started":1}
I20260929 11:00:49.483881 6349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:49.466488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35176 (local address 127.26.120.194:42157)
0929 11:00:49.476160 (+ 9672us) server_negotiation.cc:206] Beginning negotiation
0929 11:00:49.476166 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 11:00:49.476176 (+ 10us) server_negotiation.cc:407] Connection header received
0929 11:00:49.476217 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:00:49.476220 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:00:49.476237 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:00:49.476312 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:00:49.478305 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:49.479448 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:49.482185 (+ 2737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:49.483020 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:49.483114 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:49.483169 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 11:00:49.483766 (+ 597us) server_negotiation.cc:299] Negotiation successful
0929 11:00:49.483805 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9628,"thread_start_us":92,"threads_started":1}
I20260929 11:00:49.484627 6344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:49.466141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:38871 (local address 127.0.0.1:45664)
0929 11:00:49.475557 (+ 9416us) negotiation.cc:107] Waiting for socket to connect
0929 11:00:49.475571 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 11:00:49.475622 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:00:49.476790 (+ 1168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:00:49.476791 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:00:49.476792 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:00:49.476829 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:49.476831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:49.480813 (+ 3982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:49.480814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:49.482739 (+ 1925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:49.482742 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:49.484432 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:49.484433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:49.484493 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:49.484496 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 11:00:49.484526 (+ 30us) client_negotiation.cc:240] Negotiation successful
0929 11:00:49.484562 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9402,"thread_start_us":73,"threads_started":1}
I20260929 11:00:49.485162 6347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:49.466247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45664 (local address 127.26.120.195:38871)
0929 11:00:49.475757 (+ 9510us) server_negotiation.cc:206] Beginning negotiation
0929 11:00:49.475762 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 11:00:49.475771 (+ 9us) server_negotiation.cc:407] Connection header received
0929 11:00:49.475808 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:00:49.475810 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:00:49.475826 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:00:49.477005 (+ 1179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:00:49.477085 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:49.478202 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:49.482843 (+ 4641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:49.484328 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:49.484944 (+ 616us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:49.485013 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 11:00:49.485040 (+ 27us) server_negotiation.cc:299] Negotiation successful
0929 11:00:49.485085 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9472,"thread_start_us":89,"threads_started":1}
I20260929 11:00:49.485275 6131 tablet_service.cc:1467] Processing CreateTablet for tablet 0e604636b0644f9381ce8f8d703517f5 (DEFAULT_TABLE table=test-table [id=316eafd8ce894109a70e5e810b424090]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:00:49.485652 6131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e604636b0644f9381ce8f8d703517f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:00:49.485869 6266 tablet_service.cc:1467] Processing CreateTablet for tablet 0e604636b0644f9381ce8f8d703517f5 (DEFAULT_TABLE table=test-table [id=316eafd8ce894109a70e5e810b424090]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:00:49.486174 6266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e604636b0644f9381ce8f8d703517f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:00:49.489610 6353 tablet_bootstrap.cc:492] T 0e604636b0644f9381ce8f8d703517f5 P a1125bb0550545709141aacb9e0e4287: Bootstrap starting.
I20260929 11:00:49.489589 6354 tablet_bootstrap.cc:492] T 0e604636b0644f9381ce8f8d703517f5 P 0d9663cfb1be41018b883676469e802d: Bootstrap starting.
I20260929 11:00:49.490748 6354 tablet_bootstrap.cc:654] T 0e604636b0644f9381ce8f8d703517f5 P 0d9663cfb1be41018b883676469e802d: Neither blocks nor log segments found. Creating new log.
I20260929 11:00:49.491019 6353 tablet_bootstrap.cc:654] T 0e604636b0644f9381ce8f8d703517f5 P a1125bb0550545709141aacb9e0e4287: Neither blocks nor log segments found. Creating new log.
I20260929 11:00:49.491101 6354 log.cc:824] T 0e604636b0644f9381ce8f8d703517f5 P 0d9663cfb1be41018b883676469e802d: Log is configured to *not* fsync() on all Append() calls
I20260929 11:00:49.491371 6353 log.cc:824] T 0e604636b0644f9381ce8f8d703517f5 P a1125bb0550545709141aacb9e0e4287: Log is configured to *not* fsync() on all Append() calls
I20260929 11:00:49.491899 6354 tablet_bootstrap.cc:492] T 0e604636b0644f9381ce8f8d703517f5 P 0d9663cfb1be41018b883676469e802d: No bootstrap required, opened a new log
I20260929 11:00:49.491972 6354 ts_tablet_manager.cc:1397] T 0e604636b0644f9381ce8f8d703517f5 P 0d9663cfb1be41018b883676469e802d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260929 11:00:49.492100 6353 tablet_bootstrap.cc:492] T 0e604636b0644f9381ce8f8d703517f5 P a1125bb0550545709141aacb9e0e4287: No bootstrap required, opened a new log
I20260929 11:00:49.492146 6353 ts_tablet_manager.cc:1397] T 0e604636b0644f9381ce8f8d703517f5 P a1125bb0550545709141aacb9e0e4287: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 11:00:49.493983 6354 raft_consensus.cc:348] T 0e604636b0644f9381ce8f8d703517f5 P 0d9663cfb1be41018b883676469e802d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e205023eaa8343bfaed54d39d0f1941a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46329 } } peers { permanent_uuid: "0d9663cfb1be41018b883676469e802d" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 38871 } } peers { permanent_uuid: "a1125bb0550545709141aacb9e0e4287" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 42157 } }
I20260929 11:00:49.494171 6354 raft_consensus.cc:374] T 0e604636b0644f9381ce8f8d703517f5 P 0d9663cfb1be41018b883676469e802d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:00:49.494205 6354 raft_consensus.cc:729] T 0e604636b0644f9381ce8f8d703517f5 P 0d9663cfb1be41018b883676469e802d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9663cfb1be41018b883676469e802d, State: Initialized, Role: FOLLOWER
I20260929 11:00:49.494135 6353 raft_consensus.cc:348] T 0e604636b0644f9381ce8f8d703517f5 P a1125bb0550545709141aacb9e0e4287 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e205023eaa8343bfaed54d39d0f1941a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46329 } } peers { permanent_uuid: "0d9663cfb1be41018b883676469e802d" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 38871 } } peers { permanent_uuid: "a1125bb0550545709141aacb9e0e4287" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 42157 } }
I20260929 11:00:49.494267 6353 raft_consensus.cc:374] T 0e604636b0644f9381ce8f8d703517f5 P a1125bb0550545709141aacb9e0e4287 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:00:49.494299 6353 raft_consensus.cc:729] T 0e604636b0644f9381ce8f8d703517f5 P a1125bb0550545709141aacb9e0e4287 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1125bb0550545709141aacb9e0e4287, State: Initialized, Role: FOLLOWER
I20260929 11:00:49.494316 6354 consensus_queue.cc:260] T 0e604636b0644f9381ce8f8d703517f5 P 0d9663cfb1be41018b883676469e802d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e205023eaa8343bfaed54d39d0f1941a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46329 } } peers { permanent_uuid: "0d9663cfb1be41018b883676469e802d" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 38871 } } peers { permanent_uuid: "a1125bb0550545709141aacb9e0e4287" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 42157 } }
I20260929 11:00:49.494385 6353 consensus_queue.cc:260] T 0e604636b0644f9381ce8f8d703517f5 P a1125bb0550545709141aacb9e0e4287 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e205023eaa8343bfaed54d39d0f1941a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46329 } } peers { permanent_uuid: "0d9663cfb1be41018b883676469e802d" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 38871 } } peers { permanent_uuid: "a1125bb0550545709141aacb9e0e4287" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 42157 } }
I20260929 11:00:49.494621 6353 ts_tablet_manager.cc:1428] T 0e604636b0644f9381ce8f8d703517f5 P a1125bb0550545709141aacb9e0e4287: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 11:00:49.494621 6354 ts_tablet_manager.cc:1428] T 0e604636b0644f9381ce8f8d703517f5 P 0d9663cfb1be41018b883676469e802d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260929 11:00:49.494773 6197 heartbeater.cc:503] Master 127.26.120.254:44973 was elected leader, sending a full tablet report...
I20260929 11:00:49.495060 6332 heartbeater.cc:503] Master 127.26.120.254:44973 was elected leader, sending a full tablet report...
I20260929 11:00:49.525053 6352 raft_consensus.cc:482] T 0e604636b0644f9381ce8f8d703517f5 P e205023eaa8343bfaed54d39d0f1941a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 11:00:49.525249 6352 raft_consensus.cc:504] T 0e604636b0644f9381ce8f8d703517f5 P e205023eaa8343bfaed54d39d0f1941a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e205023eaa8343bfaed54d39d0f1941a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46329 } } peers { permanent_uuid: "0d9663cfb1be41018b883676469e802d" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 38871 } } peers { permanent_uuid: "a1125bb0550545709141aacb9e0e4287" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 42157 } }
I20260929 11:00:49.525695 6352 leader_election.cc:288] T 0e604636b0644f9381ce8f8d703517f5 P e205023eaa8343bfaed54d39d0f1941a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9663cfb1be41018b883676469e802d (127.26.120.195:38871), a1125bb0550545709141aacb9e0e4287 (127.26.120.194:42157)
I20260929 11:00:49.529907 6347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:49.526019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:54095 (local address 127.26.120.195:38871)
0929 11:00:49.526078 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0929 11:00:49.526080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:00:49.526225 (+ 145us) server_negotiation.cc:407] Connection header received
0929 11:00:49.526335 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:00:49.526336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:00:49.526345 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:00:49.526375 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:00:49.526643 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:49.527383 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:49.528952 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:49.529529 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:49.529574 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:49.529583 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 11:00:49.529795 (+ 212us) server_negotiation.cc:299] Negotiation successful
0929 11:00:49.529835 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 11:00:49.530359 6359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:49.525925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:38871 (local address 127.26.120.193:54095)
0929 11:00:49.526158 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0929 11:00:49.526169 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 11:00:49.526255 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:00:49.526466 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:00:49.526468 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:00:49.526482 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:00:49.526564 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:49.526566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:49.527507 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:49.527508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:49.528840 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:49.528844 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:49.529647 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:49.529648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:49.529717 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:49.529721 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 11:00:49.530223 (+ 502us) client_negotiation.cc:240] Negotiation successful
0929 11:00:49.530277 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":149,"threads_started":1}
I20260929 11:00:49.530833 6286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e604636b0644f9381ce8f8d703517f5" candidate_uuid: "e205023eaa8343bfaed54d39d0f1941a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9663cfb1be41018b883676469e802d" is_pre_election: true
I20260929 11:00:49.531002 6286 raft_consensus.cc:2454] T 0e604636b0644f9381ce8f8d703517f5 P 0d9663cfb1be41018b883676469e802d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e205023eaa8343bfaed54d39d0f1941a in term 0.
I20260929 11:00:49.531324 5948 leader_election.cc:302] T 0e604636b0644f9381ce8f8d703517f5 P e205023eaa8343bfaed54d39d0f1941a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9663cfb1be41018b883676469e802d, e205023eaa8343bfaed54d39d0f1941a; no voters:
I20260929 11:00:49.531453 6352 raft_consensus.cc:2781] T 0e604636b0644f9381ce8f8d703517f5 P e205023eaa8343bfaed54d39d0f1941a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 11:00:49.531490 6352 raft_consensus.cc:482] T 0e604636b0644f9381ce8f8d703517f5 P e205023eaa8343bfaed54d39d0f1941a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 11:00:49.531510 6352 raft_consensus.cc:3037] T 0e604636b0644f9381ce8f8d703517f5 P e205023eaa8343bfaed54d39d0f1941a [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:00:49.532375 6352 raft_consensus.cc:504] T 0e604636b0644f9381ce8f8d703517f5 P e205023eaa8343bfaed54d39d0f1941a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e205023eaa8343bfaed54d39d0f1941a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46329 } } peers { permanent_uuid: "0d9663cfb1be41018b883676469e802d" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 38871 } } peers { permanent_uuid: "a1125bb0550545709141aacb9e0e4287" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 42157 } }
I20260929 11:00:49.532512 6352 leader_election.cc:288] T 0e604636b0644f9381ce8f8d703517f5 P e205023eaa8343bfaed54d39d0f1941a [CANDIDATE]: Term 1 election: Requested vote from peers 0d9663cfb1be41018b883676469e802d (127.26.120.195:38871), a1125bb0550545709141aacb9e0e4287 (127.26.120.194:42157)
I20260929 11:00:49.532827 6286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e604636b0644f9381ce8f8d703517f5" candidate_uuid: "e205023eaa8343bfaed54d39d0f1941a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9663cfb1be41018b883676469e802d"
I20260929 11:00:49.532900 6286 raft_consensus.cc:3037] T 0e604636b0644f9381ce8f8d703517f5 P 0d9663cfb1be41018b883676469e802d [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:00:49.533870 6286 raft_consensus.cc:2454] T 0e604636b0644f9381ce8f8d703517f5 P 0d9663cfb1be41018b883676469e802d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e205023eaa8343bfaed54d39d0f1941a in term 1.
I20260929 11:00:49.534123 5948 leader_election.cc:302] T 0e604636b0644f9381ce8f8d703517f5 P e205023eaa8343bfaed54d39d0f1941a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9663cfb1be41018b883676469e802d, e205023eaa8343bfaed54d39d0f1941a; no voters:
I20260929 11:00:49.534253 6352 raft_consensus.cc:2781] T 0e604636b0644f9381ce8f8d703517f5 P e205023eaa8343bfaed54d39d0f1941a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:00:49.534406 6352 raft_consensus.cc:686] T 0e604636b0644f9381ce8f8d703517f5 P e205023eaa8343bfaed54d39d0f1941a [term 1 LEADER]: Becoming Leader. State: Replica: e205023eaa8343bfaed54d39d0f1941a, State: Running, Role: LEADER
I20260929 11:00:49.534503 6352 consensus_queue.cc:237] T 0e604636b0644f9381ce8f8d703517f5 P e205023eaa8343bfaed54d39d0f1941a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e205023eaa8343bfaed54d39d0f1941a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46329 } } peers { permanent_uuid: "0d9663cfb1be41018b883676469e802d" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 38871 } } peers { permanent_uuid: "a1125bb0550545709141aacb9e0e4287" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 42157 } }
I20260929 11:00:49.534996 6349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:49.526372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:44821 (local address 127.26.120.194:42157)
0929 11:00:49.526410 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0929 11:00:49.526412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:00:49.527444 (+ 1032us) server_negotiation.cc:407] Connection header received
0929 11:00:49.527555 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:00:49.527555 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:00:49.527565 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:00:49.527594 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:00:49.529482 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:49.530211 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:49.531821 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:49.532389 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:49.532440 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:49.532449 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 11:00:49.534861 (+ 2412us) server_negotiation.cc:299] Negotiation successful
0929 11:00:49.534930 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 11:00:49.535367 5847 catalog_manager.cc:5495] T 0e604636b0644f9381ce8f8d703517f5 P e205023eaa8343bfaed54d39d0f1941a reported cstate change: term changed from 0 to 1, leader changed from <none> to e205023eaa8343bfaed54d39d0f1941a (127.26.120.193). New cstate: current_term: 1 leader_uuid: "e205023eaa8343bfaed54d39d0f1941a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e205023eaa8343bfaed54d39d0f1941a" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 46329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d9663cfb1be41018b883676469e802d" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 38871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1125bb0550545709141aacb9e0e4287" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 42157 } health_report { overall_health: UNKNOWN } } }
I20260929 11:00:49.536149 6360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:49.526214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:42157 (local address 127.26.120.193:44821)
0929 11:00:49.527363 (+ 1149us) negotiation.cc:107] Waiting for socket to connect
0929 11:00:49.527377 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 11:00:49.527472 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:00:49.527659 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:00:49.527659 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:00:49.527661 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:00:49.527724 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:49.527725 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:49.530637 (+ 2912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:49.530638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:49.531722 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:00:49.531724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:00:49.534695 (+ 2971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:00:49.534696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:00:49.534768 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:49.534772 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 11:00:49.535099 (+ 327us) client_negotiation.cc:240] Negotiation successful
0929 11:00:49.536046 (+ 947us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1131,"thread_start_us":71,"threads_started":1}
I20260929 11:00:49.536597 6136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e604636b0644f9381ce8f8d703517f5" candidate_uuid: "e205023eaa8343bfaed54d39d0f1941a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1125bb0550545709141aacb9e0e4287" is_pre_election: true
I20260929 11:00:49.536742 6136 raft_consensus.cc:2454] T 0e604636b0644f9381ce8f8d703517f5 P a1125bb0550545709141aacb9e0e4287 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e205023eaa8343bfaed54d39d0f1941a in term 0.
I20260929 11:00:49.536844 6135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e604636b0644f9381ce8f8d703517f5" candidate_uuid: "e205023eaa8343bfaed54d39d0f1941a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1125bb0550545709141aacb9e0e4287"
I20260929 11:00:49.536918 6135 raft_consensus.cc:3037] T 0e604636b0644f9381ce8f8d703517f5 P a1125bb0550545709141aacb9e0e4287 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:00:49.537806 6135 raft_consensus.cc:2454] T 0e604636b0644f9381ce8f8d703517f5 P a1125bb0550545709141aacb9e0e4287 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e205023eaa8343bfaed54d39d0f1941a in term 1.
I20260929 11:00:49.546168 6345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:49.543624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51838 (local address 127.26.120.193:46329)
0929 11:00:49.543663 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 11:00:49.543665 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:00:49.543687 (+ 22us) server_negotiation.cc:407] Connection header received
0929 11:00:49.543767 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:00:49.543768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:00:49.543789 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:00:49.543852 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 11:00:49.544011 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:49.544482 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:49.545270 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:00:49.545483 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:00:49.545577 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:00:49.545758 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 11:00:49.545870 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 11:00:49.545964 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 11:00:49.546005 (+ 41us) server_negotiation.cc:299] Negotiation successful
0929 11:00:49.546027 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 11:00:49.548127 6286 raft_consensus.cc:1261] T 0e604636b0644f9381ce8f8d703517f5 P 0d9663cfb1be41018b883676469e802d [term 1 FOLLOWER]: Refusing update from remote peer e205023eaa8343bfaed54d39d0f1941a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 11:00:49.548100 6135 raft_consensus.cc:1261] T 0e604636b0644f9381ce8f8d703517f5 P a1125bb0550545709141aacb9e0e4287 [term 1 FOLLOWER]: Refusing update from remote peer e205023eaa8343bfaed54d39d0f1941a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 11:00:49.548501 6361 consensus_queue.cc:1035] T 0e604636b0644f9381ce8f8d703517f5 P e205023eaa8343bfaed54d39d0f1941a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1125bb0550545709141aacb9e0e4287" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 42157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 11:00:49.548689 6352 consensus_queue.cc:1035] T 0e604636b0644f9381ce8f8d703517f5 P e205023eaa8343bfaed54d39d0f1941a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9663cfb1be41018b883676469e802d" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 38871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 11:00:49.551506 6366 mvcc.cc:205] Tried to move back new op lower bound from 7334623844546121728 to 7334623844493938688. Current Snapshot: MvccSnapshot[applied={T|T < 7334623844546121728}]
I20260929 11:00:49.554184 6365 mvcc.cc:205] Tried to move back new op lower bound from 7334623844546121728 to 7334623844493938688. Current Snapshot: MvccSnapshot[applied={T|T < 7334623844546121728}]
I20260929 11:00:49.554378 5848 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55358:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 11:00:49.554474 5848 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55358:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 11:00:49.555213 5848 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 374b71e63e8b4fa188fb5c17daa8c964: Sending DeleteTablet for 3 replicas of tablet 0e604636b0644f9381ce8f8d703517f5
I20260929 11:00:49.555634 5993 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e604636b0644f9381ce8f8d703517f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:00:49 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:51834
I20260929 11:00:49.555788 6266 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e604636b0644f9381ce8f8d703517f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:00:49 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:45664
I20260929 11:00:49.555980 6375 tablet_replica.cc:330] T 0e604636b0644f9381ce8f8d703517f5 P 0d9663cfb1be41018b883676469e802d: stopping tablet replica
I20260929 11:00:49.556082 6375 raft_consensus.cc:2229] T 0e604636b0644f9381ce8f8d703517f5 P 0d9663cfb1be41018b883676469e802d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 11:00:49.556290 6131 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e604636b0644f9381ce8f8d703517f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:00:49 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:35176
I20260929 11:00:49.556665 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 5929
W20260929 11:00:49.563112 5835 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.120.193:46329 (error 108)
I20260929 11:00:49.563390 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 6066
I20260929 11:00:49.564270 6344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:49.564140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:46329 (local address 127.0.0.1:51846)
0929 11:00:49.564189 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0929 11:00:49.564228 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.193:46329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260929 11:00:49.564481 5835 catalog_manager.cc:4584] TS e205023eaa8343bfaed54d39d0f1941a (127.26.120.193:46329): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0e604636b0644f9381ce8f8d703517f5: Network error: Client connection negotiation failed: client connection to 127.26.120.193:46329: connect: Connection refused (error 111)
I20260929 11:00:49.565393 6375 raft_consensus.cc:2258] T 0e604636b0644f9381ce8f8d703517f5 P 0d9663cfb1be41018b883676469e802d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 11:00:49.566005 6375 ts_tablet_manager.cc:1905] T 0e604636b0644f9381ce8f8d703517f5 P 0d9663cfb1be41018b883676469e802d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 11:00:49.568131 6375 ts_tablet_manager.cc:1918] T 0e604636b0644f9381ce8f8d703517f5 P 0d9663cfb1be41018b883676469e802d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 11:00:49.568272 6375 log.cc:1196] T 0e604636b0644f9381ce8f8d703517f5 P 0d9663cfb1be41018b883676469e802d: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790679549986039-27107-0/minicluster-data/ts-2/wal/wals/0e604636b0644f9381ce8f8d703517f5
I20260929 11:00:49.568961 6375 ts_tablet_manager.cc:1939] T 0e604636b0644f9381ce8f8d703517f5 P 0d9663cfb1be41018b883676469e802d: Deleting consensus metadata
I20260929 11:00:49.569594 5835 catalog_manager.cc:4841] TS 0d9663cfb1be41018b883676469e802d (127.26.120.195:38871): tablet 0e604636b0644f9381ce8f8d703517f5 (table test-table [id=316eafd8ce894109a70e5e810b424090]) successfully deleted
I20260929 11:00:49.569753 6344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:00:49.569520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:42157 (local address 127.0.0.1:35188)
0929 11:00:49.569695 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0929 11:00:49.569723 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.194:42157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155}
I20260929 11:00:49.569777 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 6203
I20260929 11:00:49.574816 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 5816
I20260929 11:00:49.580084 27107 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260929 11:00:49.918282 27107 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260929 11:00:49.918413 27107 mini_ranger.cc:66] Stopping Ranger...
I20260929 11:00:50.262857 27107 mini_ranger.cc:68] Stopped Ranger
2026-09-29 11:00:50.263 UTC [5402] LOG: received smart shutdown request
2026-09-29 11:00:50.266 UTC [5402] LOG: background worker "logical replication launcher" (PID 5411) exited with exit code 1
2026-09-29 11:00:50.266 UTC [5406] LOG: shutting down
2026-09-29 11:00:50.448 UTC [5402] LOG: database system is shut down
2026-09-29T11:00:50Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (46692 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 11:00:50 dist-test-slave-2lzq krb5kdc[6386](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 11:00:50 dist-test-slave-2lzq krb5kdc[6386](info): set up 2 sockets
Sep 29 11:00:50 dist-test-slave-2lzq krb5kdc[6386](info): commencing operation
krb5kdc: starting...
W20260929 11:00:51.602262 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 11:00:51 dist-test-slave-2lzq krb5kdc[6386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T11:00:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T11:00:51Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.26.120.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.26.120.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.26.120.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/rangeradmin_127.26.120.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.26.120.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/rangeradmin_127.26.120.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.26.120.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.26.120.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.26.120.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/rangerlookup_127.26.120.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.26.120.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/rangerlookup_127.26.120.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.26.120.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.120.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.120.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/HTTP_127.26.120.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.26.120.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/HTTP_127.26.120.212@KRBTEST.COM.keytab.
I20260929 11:00:51.768814 27107 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskkJyhTP/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/postgres -l logfile start
2026-09-29 11:00:54.217 UTC [6416] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-29 11:00:54.217 UTC [6416] LOG: listening on IPv4 address "127.26.120.212", port 56833
2026-09-29 11:00:54.220 UTC [6416] LOG: listening on Unix socket "/tmp/.s.PGSQL.56833"
2026-09-29 11:00:54.254 UTC [6419] LOG: database system was shut down at 2026-09-29 11:00:52 UTC
2026-09-29 11:00:54.258 UTC [6416] LOG: database system is ready to accept connections
I20260929 11:00:55.210223 27107 mini_postgres.cc:96] Postgres bound to 56833
2026-09-29 11:00:55.216 UTC [6427] FATAL: database "slave" does not exist
127.26.120.212:56833 - accepting connections
I20260929 11:00:55.216814 27107 mini_ranger.cc:161] Starting Ranger...
I20260929 11:00:55.231638 27107 mini_ranger.cc:84] Created miniranger Postgres user
I20260929 11:00:55.711444 27107 mini_ranger.cc:87] Created ranger Postgres database
I20260929 11:00:55.711544 27107 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ranger-admin
2026-09-29 11:00:55,985 [I] DB FLAVOR :POSTGRES
2026-09-29 11:00:55,985 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 11:00:55,985 [I] Checking connection..
2026-09-29 11:00:55,985 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:56833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-29 11:00:56,254 [I] Checking connection passed.
2026-09-29 11:00:56,254 [I] --------- Verifying version history table ---------
2026-09-29 11:00:56,254 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:56833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 11:00:56,517 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 11:00:56,518 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:56833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 11:00:56,784 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 11:00:56,784 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-29 11:00:56,784 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:56833/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-29 11:00:57.031 UTC [6491] WARNING: there is no transaction in progress
2026-09-29 11:00:57,038 [I] create_dbversion_catalog.sql file imported successfully
2026-09-29 11:00:57,038 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:56833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 11:00:57,282 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-29 11:00:57,282 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 11:00:57,282 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:56833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-29 11:00:57,538 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:56833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-29 11:00:57,786 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:56833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-2lzq.c.gcp-upstream.internal','N') ;"
2026-09-29 11:00:58,014 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:56833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 11:00:58,282 [I] Table x_portal_user does not exist in database ranger
2026-09-29 11:00:58,282 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:56833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 11:00:58,538 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-29 11:00:58,538 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:56833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 11:00:58,794 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:56833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 11:00:59,040 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-29 11:00:59,040 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:56833/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-29 11:00:59.991 UTC [6599] WARNING: there is no transaction in progress
2026-09-29 11:01:00.006 UTC [6599] WARNING: there is no transaction in progress
2026-09-29 11:01:00.022 UTC [6599] WARNING: there is no transaction in progress
2026-09-29 11:01:00.037 UTC [6599] WARNING: there is no transaction in progress
2026-09-29 11:01:00.052 UTC [6599] WARNING: there is no transaction in progress
2026-09-29 11:01:00.072 UTC [6599] WARNING: there is no transaction in progress
2026-09-29 11:01:00.130 UTC [6599] WARNING: there is no transaction in progress
2026-09-29 11:01:00.137 UTC [6599] WARNING: there is no transaction in progress
2026-09-29 11:01:00.152 UTC [6599] WARNING: there is no transaction in progress
2026-09-29 11:01:00.162 UTC [6599] WARNING: there is no transaction in progress
2026-09-29 11:01:00.174 UTC [6599] WARNING: there is no transaction in progress
2026-09-29 11:01:00.188 UTC [6599] WARNING: there is no transaction in progress
2026-09-29 11:01:00.198 UTC [6599] WARNING: there is no transaction in progress
2026-09-29 11:01:00.205 UTC [6599] WARNING: there is no transaction in progress
2026-09-29 11:01:00.212 UTC [6599] WARNING: there is no transaction in progress
2026-09-29 11:01:00.219 UTC [6599] WARNING: there is no transaction in progress
2026-09-29 11:01:00,902 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-29 11:01:00,903 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:56833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-29 11:01:01,125 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-29 11:01:01,126 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:56833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 11:01:01,361 [I] Table x_portal_user already exists in database 'ranger'
2026-09-29 11:01:01,362 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:56833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 11:01:01,626 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-29 11:01:01,626 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:56833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 11:01:01,859 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:56833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 11:01:02,094 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:56833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-2lzq.c.gcp-upstream.internal';"
2026-09-29 11:01:02,341 [I] CORE_DB_SCHEMA import status has been updated
2026-09-29 11:01:02,341 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:56833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-29 11:01:02,593 [I] DB_PATCHES have already been applied
I20260929 11:01:02.599086 27107 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ranger-admin:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskkJyhTP/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260929 11:01:02.599151 27107 mini_ranger.cc:193] Using host: 127.26.120.212
I20260929 11:01:02.602387 27107 mini_ranger.cc:227] Ranger admin URL: http://127.26.120.212:58097
Sep 29, 2026 11:01:02 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 11:01:02 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 11:01:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ranger-admin/ews/webapp
Sep 29, 2026 11:01:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 29, 2026 11:01:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 29, 2026 11:01:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ranger-admin/ews/webapp].
Sep 29, 2026 11:01:03 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260929 11:01:03.712790 27107 mini_ranger.cc:160] Time spent starting Ranger: real 8.496s user 0.001s sys 0.008s
Sep 29, 2026 11:01:24 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 29, 2026 11:01:24 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 29, 2026 11:01:24 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 29, 2026 11:01:24 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 29, 2026 11:01:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 29, 2026 11:01:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 29, 2026 11:01:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 29, 2026 11:01:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 29, 2026 11:01:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 29, 2026 11:01:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 29, 2026 11:01:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 29, 2026 11:01:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 29, 2026 11:01:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 29, 2026 11:01:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 29, 2026 11:01:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 29, 2026 11:01:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 29, 2026 11:01:24 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 29, 2026 11:01:25 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260929 11:01:25.680709 27107 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.26.120.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.26.120.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.26.120.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/rangerkms_127.26.120.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.26.120.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/rangerkms_127.26.120.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.26.120.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.120.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.120.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/HTTP_127.26.120.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.26.120.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/HTTP_127.26.120.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 29 11:01:25 dist-test-slave-2lzq krb5kdc[6386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679685, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260929 11:01:25.765672 27107 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260929 11:01:25.774601 27107 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260929 11:01:26.606891 27107 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260929 11:01:26.606986 27107 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ranger-kms
I20260929 11:01:26.607007 27107 mini_ranger_kms.cc:227] Using postgres at 127.26.120.212:56833
2026-09-29 11:01:26,772 [I] DB FLAVOR :POSTGRES
2026-09-29 11:01:26,773 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 11:01:26,773 [I] Checking connection
2026-09-29 11:01:26,773 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:56833/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-29 11:01:27,000 [I] connection success
2026-09-29 11:01:27,000 [I] --------- Verifying Ranger DB tables ---------
2026-09-29 11:01:27,000 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-29 11:01:27,000 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:56833/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-29 11:01:27,246 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-29 11:01:27,246 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 11:01:27,247 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-29 11:01:27,247 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.120.212:56833/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-29 11:01:27,494 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260929 11:01:27.988294 27107 mini_ranger_kms.cc:310] Created kms service
I20260929 11:01:28.161967 27107 mini_ranger_kms.cc:326] Created kudu user
I20260929 11:01:28.209395 27107 mini_ranger_kms.cc:343] Created rangerkms user
Sep 29 11:01:28 dist-test-slave-2lzq krb5kdc[6386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679685, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.26.120.212@KRBTEST.COM
I20260929 11:01:28.575992 27107 mini_ranger_kms.cc:384] Added ranger policy
I20260929 11:01:28.576196 27107 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ranger-kms:/tmp/dist-test-taskkJyhTP/build/release/bin/postgresql.jar:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskkJyhTP/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260929 11:01:28.576222 27107 mini_ranger_kms.cc:240] Using host: 127.26.120.212
I20260929 11:01:28.579681 27107 mini_ranger_kms.cc:276] Ranger KMS PID: 6779
I20260929 11:01:28.579766 27107 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.26.120.212:60499
Sep 29, 2026 11:01:29 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 11:01:29 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 11:01:29 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 29, 2026 11:01:29 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 29, 2026 11:01:30 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ranger-kms/ews/webapp].
Sep 29, 2026 11:01:30 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Sep 29 11:01:30 dist-test-slave-2lzq krb5kdc[6386](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790679690, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.120.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:01:30.833510 27107 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.068s user 0.005s sys 0.143s
I20260929 11:01:30.833621 27107 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260929 11:01:30.833673 27107 mini_ranger_kms.cc:401] 127.26.120.212:60499/kms/v1/keys
Sep 29 11:01:31 dist-test-slave-2lzq krb5kdc[6386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679690, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.120.212@KRBTEST.COM for HTTP/127.26.120.212@KRBTEST.COM
Sep 29 11:01:31 dist-test-slave-2lzq krb5kdc[6386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679690, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.120.212@KRBTEST.COM for HTTP/127.26.120.212@KRBTEST.COM
Sep 29 11:01:33 dist-test-slave-2lzq krb5kdc[6386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.26.120.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
I20260929 11:01:33.870846 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:36651
--webserver_interface=127.26.120.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:43793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:36651
--ranger_config_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.120.212:60499/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:01:33.952384 6830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:01:33.952580 6830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:01:33.952601 6830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:01:33.953768 6830 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:01:33.953811 6830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:01:33.953825 6830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:01:33.953838 6830 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:01:33.953850 6830 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:01:33.955513 6830 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:43793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/master-0/wal
--ranger_kms_url=127.26.120.212:60499/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:36651
--ranger_config_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:36651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:01:33.955750 6830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:01:33.955992 6830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:01:33.961073 6836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:01:33.961135 6830 server_base.cc:1034] running on GCE node
W20260929 11:01:33.961165 6838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:01:33.961086 6835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:01:33.961542 6830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:01:33.961858 6830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:01:33.963045 6830 hybrid_clock.cc:648] HybridClock initialized: now 1790679693963010 us; error 51 us; skew 500 ppm
Sep 29 11:01:33 dist-test-slave-2lzq krb5kdc[6386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679693, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:01:33.969832 6830 init.cc:377] Logged in from keytab as kudu/127.26.120.254@KRBTEST.COM (short username kudu)
I20260929 11:01:33.970395 6830 webserver.cc:460] Webserver started at http://127.26.120.254:44409/ using document root <none> and password file <none>
I20260929 11:01:33.970636 6830 fs_manager.cc:359] Metadata directory not provided
I20260929 11:01:33.970695 6830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:01:33.970845 6830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 11:01:33 dist-test-slave-2lzq krb5kdc[6386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679693, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for HTTP/127.26.120.212@KRBTEST.COM
I20260929 11:01:34.016590 6830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "78c57253136c43fb9912ee92018555b4"
format_stamp: "Formatted at 2026-09-29 11:01:34 on dist-test-slave-2lzq"
server_key: "aa64d7ac437dd2ad986166786ca13d7e"
server_key_iv: "d84d1d2a8dc9d107381d64721c9bde01"
server_key_version: "kuduclusterkey@0"
I20260929 11:01:34.016999 6830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "78c57253136c43fb9912ee92018555b4"
format_stamp: "Formatted at 2026-09-29 11:01:34 on dist-test-slave-2lzq"
server_key: "aa64d7ac437dd2ad986166786ca13d7e"
server_key_iv: "d84d1d2a8dc9d107381d64721c9bde01"
server_key_version: "kuduclusterkey@0"
I20260929 11:01:34.018541 6830 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 11:01:34.030196 6845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:34.030424 6830 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:01:34.030526 6830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "78c57253136c43fb9912ee92018555b4"
format_stamp: "Formatted at 2026-09-29 11:01:34 on dist-test-slave-2lzq"
server_key: "aa64d7ac437dd2ad986166786ca13d7e"
server_key_iv: "d84d1d2a8dc9d107381d64721c9bde01"
server_key_version: "kuduclusterkey@0"
I20260929 11:01:34.030638 6830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:01:34.047868 6830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:01:34.049990 6830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:01:34.050180 6830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:01:34.054243 6830 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:36651
I20260929 11:01:34.054304 6897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:36651 every 8 connection(s)
I20260929 11:01:34.054680 6830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:01:34.055331 6898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:01:34.056195 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 6830
I20260929 11:01:34.056274 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 11:01:34.056445 27107 external_mini_cluster.cc:1428] Setting key 804efd866957f887b24b4c52468b1754
I20260929 11:01:34.058491 6898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4: Bootstrap starting.
I20260929 11:01:34.059095 6898 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4: Neither blocks nor log segments found. Creating new log.
I20260929 11:01:34.059376 6898 log.cc:824] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4: Log is configured to *not* fsync() on all Append() calls
I20260929 11:01:34.060014 6898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4: No bootstrap required, opened a new log
I20260929 11:01:34.061744 6898 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78c57253136c43fb9912ee92018555b4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 36651 } }
I20260929 11:01:34.061895 6898 raft_consensus.cc:374] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:01:34.061942 6898 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78c57253136c43fb9912ee92018555b4, State: Initialized, Role: FOLLOWER
I20260929 11:01:34.062109 6898 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78c57253136c43fb9912ee92018555b4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 36651 } }
I20260929 11:01:34.062201 6898 raft_consensus.cc:388] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 11:01:34.062244 6898 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 11:01:34.062291 6898 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:01:34.062923 6898 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78c57253136c43fb9912ee92018555b4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 36651 } }
I20260929 11:01:34.063047 6898 leader_election.cc:302] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 78c57253136c43fb9912ee92018555b4; no voters:
I20260929 11:01:34.063202 6898 leader_election.cc:288] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4 [CANDIDATE]: Term 1 election: Requested vote from peers
Sep 29 11:01:34 dist-test-slave-2lzq krb5kdc[6386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:01:34.063297 6903 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:01:34.063510 6903 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4 [term 1 LEADER]: Becoming Leader. State: Replica: 78c57253136c43fb9912ee92018555b4, State: Running, Role: LEADER
I20260929 11:01:34.063524 6898 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:01:34.063655 6903 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78c57253136c43fb9912ee92018555b4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 36651 } }
I20260929 11:01:34.064033 6904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "78c57253136c43fb9912ee92018555b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78c57253136c43fb9912ee92018555b4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 36651 } } }
I20260929 11:01:34.064112 6904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4 [sys.catalog]: This master's current role is: LEADER
I20260929 11:01:34.064393 6905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78c57253136c43fb9912ee92018555b4. Latest consensus state: current_term: 1 leader_uuid: "78c57253136c43fb9912ee92018555b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78c57253136c43fb9912ee92018555b4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 36651 } } }
I20260929 11:01:34.064477 6905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4 [sys.catalog]: This master's current role is: LEADER
I20260929 11:01:34.064851 6909 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:01:34.065137 6898 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260929 11:01:34.065497 6909 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:01:34.070619 6901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:34.056971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36320 (local address 127.26.120.254:36651)
0929 11:01:34.057161 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:34.057170 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:34.057203 (+ 33us) server_negotiation.cc:407] Connection header received
0929 11:01:34.057697 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:34.057706 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:34.057779 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:34.057938 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:01:34.058648 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:34.059691 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:34.060525 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:34.060937 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:34.061089 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:34.063862 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:01:34.063866 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:01:34.063866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:01:34.063878 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:01:34.065924 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:34.067385 (+ 1461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:34.067440 (+ 55us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:34.067444 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:34.068013 (+ 569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:34.068309 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:34.068313 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:34.068314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:34.069124 (+ 810us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:01:34.069470 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:34.069613 (+ 143us) server_negotiation.cc:299] Negotiation successful
0929 11:01:34.070209 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":82,"threads_started":1}
I20260929 11:01:34.070783 6909 catalog_manager.cc:1347] Generated new cluster ID: e8de7e20562545218fb46906ec218742
I20260929 11:01:34.070858 6909 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 11:01:34.102452 6909 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 11:01:34.103045 6909 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:01:34.112221 6909 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4: Generated new TSK 0
I20260929 11:01:34.112578 6909 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.120.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab.
Entry for principal kudu/127.26.120.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab.
I20260929 11:01:34.812985 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.120.212:60499/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.193:0
--local_ip_for_outbound_sockets=127.26.120.193
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:36651
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:43793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:01:34.929972 6943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:01:34.930235 6943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:01:34.930258 6943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:01:34.931448 6943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:01:34.931491 6943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:01:34.931519 6943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.193
I20260929 11:01:34.933569 6943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:43793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.26.120.212:60499/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:36651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.193
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:01:34.933812 6943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:01:34.934101 6943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:01:34.940142 6951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:01:34.940198 6943 server_base.cc:1034] running on GCE node
W20260929 11:01:34.940336 6948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:01:34.940183 6949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:01:34.940647 6943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:01:34.940923 6943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:01:34.942098 6943 hybrid_clock.cc:648] HybridClock initialized: now 1790679694942076 us; error 36 us; skew 500 ppm
Sep 29 11:01:34 dist-test-slave-2lzq krb5kdc[6386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679694, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:01:34.944772 6943 init.cc:377] Logged in from keytab as kudu/127.26.120.193@KRBTEST.COM (short username kudu)
I20260929 11:01:34.945307 6943 webserver.cc:460] Webserver started at http://127.26.120.193:42839/ using document root <none> and password file <none>
I20260929 11:01:34.945565 6943 fs_manager.cc:359] Metadata directory not provided
I20260929 11:01:34.945612 6943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:01:34.945742 6943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 11:01:34 dist-test-slave-2lzq krb5kdc[6386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679694, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for HTTP/127.26.120.212@KRBTEST.COM
I20260929 11:01:34.958993 6943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-0/data/instance:
uuid: "1a66282683364798a21e557afdedd9e0"
format_stamp: "Formatted at 2026-09-29 11:01:34 on dist-test-slave-2lzq"
server_key: "6855a139b8a4cbaa1bd90f554c3447cc"
server_key_iv: "15e833791a6aed400e15d9706b9ec47a"
server_key_version: "kuduclusterkey@0"
I20260929 11:01:34.959456 6943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance:
uuid: "1a66282683364798a21e557afdedd9e0"
format_stamp: "Formatted at 2026-09-29 11:01:34 on dist-test-slave-2lzq"
server_key: "6855a139b8a4cbaa1bd90f554c3447cc"
server_key_iv: "15e833791a6aed400e15d9706b9ec47a"
server_key_version: "kuduclusterkey@0"
I20260929 11:01:34.962182 6943 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260929 11:01:34.980432 6960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:34.980683 6943 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260929 11:01:34.980794 6943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-0/wal
uuid: "1a66282683364798a21e557afdedd9e0"
format_stamp: "Formatted at 2026-09-29 11:01:34 on dist-test-slave-2lzq"
server_key: "6855a139b8a4cbaa1bd90f554c3447cc"
server_key_iv: "15e833791a6aed400e15d9706b9ec47a"
server_key_version: "kuduclusterkey@0"
I20260929 11:01:34.980901 6943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:01:35.000801 6943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:01:35.003401 6943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:01:35.003640 6943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:01:35.003926 6943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:01:35.004314 6943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:01:35.004412 6943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:35.004504 6943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:01:35.004587 6943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:35.010918 6943 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.193:42437
I20260929 11:01:35.011479 6943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
I20260929 11:01:35.013216 7073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.193:42437 every 8 connection(s)
I20260929 11:01:35.016680 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 6943
I20260929 11:01:35.016767 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance
I20260929 11:01:35.016935 27107 external_mini_cluster.cc:1428] Setting key 427f8b13928ee18031f3257f661e6de6
Sep 29 11:01:35 dist-test-slave-2lzq krb5kdc[6386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679694, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:01:35.024334 7077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:35.013857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:36651 (local address 127.26.120.193:33739)
0929 11:01:35.014119 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:35.014165 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:35.014679 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:35.014961 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:35.014970 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:35.015153 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:01:35.015347 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:35.015351 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:35.016085 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:35.016085 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:35.017243 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:35.017246 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:35.017803 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:35.017804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:35.017968 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:35.018590 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:01:35.018601 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:01:35.020332 (+ 1731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:01:35.022704 (+ 2372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:35.022711 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:35.022713 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:35.022995 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:35.023214 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:35.023215 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:35.023215 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:35.023342 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:35.023641 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:01:35.023647 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:01:35.023808 (+ 161us) client_negotiation.cc:769] Sending connection context
0929 11:01:35.023924 (+ 116us) client_negotiation.cc:240] Negotiation successful
0929 11:01:35.024099 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":91,"threads_started":1}
I20260929 11:01:35.024694 7076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:35.013866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:33739 (local address 127.26.120.254:36651)
0929 11:01:35.014040 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:35.014042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:35.014260 (+ 218us) server_negotiation.cc:407] Connection header received
0929 11:01:35.014791 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:35.014792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:35.014806 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:35.014886 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:01:35.015461 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:35.015971 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:35.017355 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:35.017672 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:35.017732 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:35.020440 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:01:35.020441 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:01:35.020442 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:01:35.020450 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:01:35.022607 (+ 2157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:35.023083 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:35.023084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:35.023085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:35.023148 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:35.023408 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:35.023408 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:35.023409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:35.023554 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:01:35.023614 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:35.024478 (+ 864us) server_negotiation.cc:299] Negotiation successful
0929 11:01:35.024595 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":98,"threads_started":1}
I20260929 11:01:35.025273 7074 heartbeater.cc:347] Connected to a master server at 127.26.120.254:36651
I20260929 11:01:35.025368 7074 heartbeater.cc:464] Registering TS with master...
I20260929 11:01:35.025566 7074 heartbeater.cc:511] Master 127.26.120.254:36651 requested a full tablet report, sending...
I20260929 11:01:35.026191 6862 ts_manager.cc:194] Registered new tserver with Master: 1a66282683364798a21e557afdedd9e0 (127.26.120.193:42437)
I20260929 11:01:35.026897 6862 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.193@KRBTEST.COM'} at 127.26.120.193:33739
WARNING: no policy specified for kudu/127.26.120.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab.
Entry for principal kudu/127.26.120.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab.
I20260929 11:01:35.068981 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.120.212:60499/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.194:0
--local_ip_for_outbound_sockets=127.26.120.194
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:36651
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:43793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 11:01:35 dist-test-slave-2lzq krb5kdc[6386](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790679695, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260929 11:01:35.202342 7082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:01:35.202617 7082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:01:35.202648 7082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:01:35.204301 7082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:01:35.204350 7082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:01:35.204386 7082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.194
I20260929 11:01:35.206480 7082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:43793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.26.120.212:60499/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:36651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.194
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:01:35.206722 7082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:01:35.206967 7082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:01:35.212318 7087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:01:35.212306 7090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:01:35.212294 7088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:01:35.213203 7082 server_base.cc:1034] running on GCE node
I20260929 11:01:35.213454 7082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:01:35.213696 7082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:01:35.214855 7082 hybrid_clock.cc:648] HybridClock initialized: now 1790679695214841 us; error 30 us; skew 500 ppm
Sep 29 11:01:35 dist-test-slave-2lzq krb5kdc[6386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679695, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:01:35.217319 7082 init.cc:377] Logged in from keytab as kudu/127.26.120.194@KRBTEST.COM (short username kudu)
I20260929 11:01:35.217813 7082 webserver.cc:460] Webserver started at http://127.26.120.194:33339/ using document root <none> and password file <none>
I20260929 11:01:35.218128 7082 fs_manager.cc:359] Metadata directory not provided
I20260929 11:01:35.218197 7082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:01:35.218380 7082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 11:01:35 dist-test-slave-2lzq krb5kdc[6386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679695, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for HTTP/127.26.120.212@KRBTEST.COM
I20260929 11:01:35.245565 7082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-1/data/instance:
uuid: "a3254e2647ae4f26b657a1b684a252e1"
format_stamp: "Formatted at 2026-09-29 11:01:35 on dist-test-slave-2lzq"
server_key: "7c97f8ac7fdc8e95b2932bc66b0926a4"
server_key_iv: "1a6ab514024eea3716635225c0b8670a"
server_key_version: "kuduclusterkey@0"
I20260929 11:01:35.246193 7082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance:
uuid: "a3254e2647ae4f26b657a1b684a252e1"
format_stamp: "Formatted at 2026-09-29 11:01:35 on dist-test-slave-2lzq"
server_key: "7c97f8ac7fdc8e95b2932bc66b0926a4"
server_key_iv: "1a6ab514024eea3716635225c0b8670a"
server_key_version: "kuduclusterkey@0"
I20260929 11:01:35.248179 7082 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260929 11:01:35.271270 7098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:35.271472 7082 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 11:01:35.271577 7082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-1/wal
uuid: "a3254e2647ae4f26b657a1b684a252e1"
format_stamp: "Formatted at 2026-09-29 11:01:35 on dist-test-slave-2lzq"
server_key: "7c97f8ac7fdc8e95b2932bc66b0926a4"
server_key_iv: "1a6ab514024eea3716635225c0b8670a"
server_key_version: "kuduclusterkey@0"
I20260929 11:01:35.271678 7082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:01:35.309824 7082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:01:35.312408 7082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:01:35.312594 7082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:01:35.312861 7082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:01:35.313248 7082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:01:35.313300 7082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:35.313335 7082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:01:35.313397 7082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:35.320842 7082 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.194:33827
I20260929 11:01:35.321326 7082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
I20260929 11:01:35.321903 7211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.194:33827 every 8 connection(s)
I20260929 11:01:35.328096 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 7082
I20260929 11:01:35.328207 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance
I20260929 11:01:35.328390 27107 external_mini_cluster.cc:1428] Setting key 56bdd28655f6a4bf98b901ec41230c8e
Sep 29 11:01:35 dist-test-slave-2lzq krb5kdc[6386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679695, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:01:35.341184 7076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:35.327744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:58151 (local address 127.26.120.254:36651)
0929 11:01:35.327793 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:35.327795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:35.328770 (+ 975us) server_negotiation.cc:407] Connection header received
0929 11:01:35.331285 (+ 2515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:35.331287 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:35.331305 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:35.331338 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:01:35.332207 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:35.333008 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:35.334693 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:35.335009 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:35.335057 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:35.337749 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:01:35.337750 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:01:35.337751 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:01:35.337758 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:01:35.339542 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:35.340045 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:35.340046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:35.340046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:35.340108 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:35.340402 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:35.340403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:35.340403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:35.340558 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:01:35.340604 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:35.340990 (+ 386us) server_negotiation.cc:299] Negotiation successful
0929 11:01:35.341082 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 11:01:35.341940 7214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:35.327376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:36651 (local address 127.26.120.194:58151)
0929 11:01:35.327872 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:35.328679 (+ 807us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:35.331139 (+ 2460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:35.331550 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:35.331555 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:35.331806 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:01:35.332091 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:35.332097 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:35.333149 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:35.333150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:35.334590 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:35.334593 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:35.335161 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:35.335161 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:35.335300 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:35.335920 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:01:35.335930 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:01:35.337640 (+ 1710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:01:35.339641 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:35.339647 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:35.339649 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:35.339949 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:35.340182 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:35.340183 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:35.340183 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:35.340321 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:35.340654 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:01:35.340660 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:01:35.340825 (+ 165us) client_negotiation.cc:769] Sending connection context
0929 11:01:35.341539 (+ 714us) client_negotiation.cc:240] Negotiation successful
0929 11:01:35.341706 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":140,"threads_started":1}
I20260929 11:01:35.342777 7212 heartbeater.cc:347] Connected to a master server at 127.26.120.254:36651
I20260929 11:01:35.342875 7212 heartbeater.cc:464] Registering TS with master...
I20260929 11:01:35.343070 7212 heartbeater.cc:511] Master 127.26.120.254:36651 requested a full tablet report, sending...
I20260929 11:01:35.343420 6862 ts_manager.cc:194] Registered new tserver with Master: a3254e2647ae4f26b657a1b684a252e1 (127.26.120.194:33827)
I20260929 11:01:35.343888 6862 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.194@KRBTEST.COM'} at 127.26.120.194:58151
WARNING: no policy specified for kudu/127.26.120.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab.
Entry for principal kudu/127.26.120.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab.
I20260929 11:01:35.373495 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.120.212:60499/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.195:0
--local_ip_for_outbound_sockets=127.26.120.195
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:36651
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:43793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:01:35.502955 7217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:01:35.503326 7217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:01:35.503417 7217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:01:35.505281 7217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:01:35.505538 7217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:01:35.505640 7217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.195
I20260929 11:01:35.507920 7217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:43793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.26.120.212:60499/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:36651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.195
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:01:35.508452 7217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:01:35.508808 7217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:01:35.515554 7223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:01:35.516299 7222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:01:35.516602 7217 server_base.cc:1034] running on GCE node
W20260929 11:01:35.516883 7225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:01:35.517174 7217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:01:35.517488 7217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:01:35.520736 7217 hybrid_clock.cc:648] HybridClock initialized: now 1790679695520711 us; error 36 us; skew 500 ppm
Sep 29 11:01:35 dist-test-slave-2lzq krb5kdc[6386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679695, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:01:35.524093 7217 init.cc:377] Logged in from keytab as kudu/127.26.120.195@KRBTEST.COM (short username kudu)
I20260929 11:01:35.524612 7217 webserver.cc:460] Webserver started at http://127.26.120.195:44713/ using document root <none> and password file <none>
I20260929 11:01:35.524856 7217 fs_manager.cc:359] Metadata directory not provided
I20260929 11:01:35.524905 7217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:01:35.525030 7217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 11:01:35 dist-test-slave-2lzq krb5kdc[6386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679695, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for HTTP/127.26.120.212@KRBTEST.COM
I20260929 11:01:35.539592 7217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-2/data/instance:
uuid: "fb78903cc5f54fb98319ef03c29cde32"
format_stamp: "Formatted at 2026-09-29 11:01:35 on dist-test-slave-2lzq"
server_key: "151f153cb3643b846aa00dee49ca387c"
server_key_iv: "fd7d9bfc7e3afb83f0a656d6ff5853a2"
server_key_version: "kuduclusterkey@0"
I20260929 11:01:35.540253 7217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance:
uuid: "fb78903cc5f54fb98319ef03c29cde32"
format_stamp: "Formatted at 2026-09-29 11:01:35 on dist-test-slave-2lzq"
server_key: "151f153cb3643b846aa00dee49ca387c"
server_key_iv: "fd7d9bfc7e3afb83f0a656d6ff5853a2"
server_key_version: "kuduclusterkey@0"
I20260929 11:01:35.544234 7217 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.000s
I20260929 11:01:35.567126 7232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:35.567394 7217 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260929 11:01:35.567492 7217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-2/wal
uuid: "fb78903cc5f54fb98319ef03c29cde32"
format_stamp: "Formatted at 2026-09-29 11:01:35 on dist-test-slave-2lzq"
server_key: "151f153cb3643b846aa00dee49ca387c"
server_key_iv: "fd7d9bfc7e3afb83f0a656d6ff5853a2"
server_key_version: "kuduclusterkey@0"
I20260929 11:01:35.567587 7217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:01:35.602537 7217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:01:35.604967 7217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:01:35.605193 7217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:01:35.605569 7217 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:01:35.606042 7217 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:01:35.606172 7217 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:35.606249 7217 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:01:35.606304 7217 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:35.612977 7217 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.195:46027
I20260929 11:01:35.613482 7217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
I20260929 11:01:35.615367 7345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.195:46027 every 8 connection(s)
I20260929 11:01:35.623241 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 7217
I20260929 11:01:35.623356 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance
Sep 29 11:01:35 dist-test-slave-2lzq krb5kdc[6386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679695, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:01:35.623565 27107 external_mini_cluster.cc:1428] Setting key 3f353f16994e11ae408a27c463e01256
I20260929 11:01:35.627125 7076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:35.616618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:55405 (local address 127.26.120.254:36651)
0929 11:01:35.616663 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:35.616665 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:35.616938 (+ 273us) server_negotiation.cc:407] Connection header received
0929 11:01:35.617657 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:35.617658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:35.617671 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:35.617701 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:01:35.618663 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:35.619492 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:35.620933 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:35.621225 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:35.621265 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:35.623954 (+ 2689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:01:35.623956 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:01:35.623956 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:01:35.623964 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:01:35.625618 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:35.626118 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:35.626119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:35.626120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:35.626174 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:35.626427 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:35.626428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:35.626428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:35.626558 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:01:35.626606 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:35.626959 (+ 353us) server_negotiation.cc:299] Negotiation successful
0929 11:01:35.627037 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 11:01:35.627872 7348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:35.616337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:36651 (local address 127.26.120.195:55405)
0929 11:01:35.616795 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:35.616841 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:35.617530 (+ 689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:35.617911 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:35.617916 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:35.618211 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:01:35.618539 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:35.618545 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:35.619603 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:35.619604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:35.620821 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:35.620824 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:35.621365 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:35.621366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:35.621509 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:35.622204 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:01:35.622215 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:01:35.623833 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:01:35.625709 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:35.625717 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:35.625719 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:35.626002 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:35.626237 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:35.626238 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:35.626238 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:35.626362 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:35.626635 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:01:35.626641 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:01:35.626803 (+ 162us) client_negotiation.cc:769] Sending connection context
0929 11:01:35.627477 (+ 674us) client_negotiation.cc:240] Negotiation successful
0929 11:01:35.627645 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":83,"threads_started":1}
I20260929 11:01:35.628476 7346 heartbeater.cc:347] Connected to a master server at 127.26.120.254:36651
I20260929 11:01:35.628567 7346 heartbeater.cc:464] Registering TS with master...
I20260929 11:01:35.628749 7346 heartbeater.cc:511] Master 127.26.120.254:36651 requested a full tablet report, sending...
I20260929 11:01:35.629140 6862 ts_manager.cc:194] Registered new tserver with Master: fb78903cc5f54fb98319ef03c29cde32 (127.26.120.195:46027)
I20260929 11:01:35.629601 6862 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.195@KRBTEST.COM'} at 127.26.120.195:55405
I20260929 11:01:35.636312 27107 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 11:01:35.638029 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 6943
I20260929 11:01:35.646166 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 7082
I20260929 11:01:35.652550 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 7217
I20260929 11:01:35.657907 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 6830
I20260929 11:01:35.662912 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:36651
--webserver_interface=127.26.120.254
--webserver_port=44409
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:43793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:36651
--ranger_config_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.120.212:60499/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.26.120.212:60499/kms with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:01:35.747385 7349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:01:35.747581 7349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:01:35.747603 7349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:01:35.748803 7349 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:01:35.748847 7349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:01:35.748862 7349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:01:35.748914 7349 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:01:35.748937 7349 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:01:35.750541 7349 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:43793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.26.120.212:60499/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:36651
--ranger_config_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:36651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=44409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:01:35.750769 7349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:01:35.750990 7349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:01:35.755542 7355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:01:35.755698 7349 server_base.cc:1034] running on GCE node
W20260929 11:01:35.755558 7357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:01:35.755534 7354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:01:35.755990 7349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:01:35.756279 7349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:01:35.757445 7349 hybrid_clock.cc:648] HybridClock initialized: now 1790679695757441 us; error 35 us; skew 500 ppm
Sep 29 11:01:35 dist-test-slave-2lzq krb5kdc[6386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679695, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:01:35.759846 7349 init.cc:377] Logged in from keytab as kudu/127.26.120.254@KRBTEST.COM (short username kudu)
I20260929 11:01:35.760278 7349 webserver.cc:460] Webserver started at http://127.26.120.254:44409/ using document root <none> and password file <none>
I20260929 11:01:35.760473 7349 fs_manager.cc:359] Metadata directory not provided
I20260929 11:01:35.760520 7349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 11:01:35 dist-test-slave-2lzq krb5kdc[6386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679695, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for HTTP/127.26.120.212@KRBTEST.COM
I20260929 11:01:35.807791 7349 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260929 11:01:35.808619 7365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:35.808876 7349 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:01:35.809000 7349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "78c57253136c43fb9912ee92018555b4"
format_stamp: "Formatted at 2026-09-29 11:01:34 on dist-test-slave-2lzq"
server_key: "aa64d7ac437dd2ad986166786ca13d7e"
server_key_iv: "d84d1d2a8dc9d107381d64721c9bde01"
server_key_version: "kuduclusterkey@0"
I20260929 11:01:35.809412 7349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:01:35.823426 7349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:01:35.825162 7349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:01:35.825332 7349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:01:35.829487 7349 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:36651
I20260929 11:01:35.829533 7417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:36651 every 8 connection(s)
I20260929 11:01:35.829929 7349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:01:35.831144 7418 sys_catalog.cc:263] Verifying existing consensus state
I20260929 11:01:35.831807 7418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4: Bootstrap starting.
I20260929 11:01:35.838533 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 7349
I20260929 11:01:35.838884 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.120.212:60499/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.193:42437
--local_ip_for_outbound_sockets=127.26.120.193
--tserver_master_addrs=127.26.120.254:36651
--webserver_port=42839
--webserver_interface=127.26.120.193
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:43793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.26.120.212:60499/kms with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 11:01:35.840116 7418 log.cc:824] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4: Log is configured to *not* fsync() on all Append() calls
I20260929 11:01:35.841871 7418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 11:01:35.842229 7418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4: Bootstrap complete.
I20260929 11:01:35.844652 7418 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78c57253136c43fb9912ee92018555b4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 36651 } }
I20260929 11:01:35.844883 7418 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78c57253136c43fb9912ee92018555b4, State: Initialized, Role: FOLLOWER
I20260929 11:01:35.845050 7418 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78c57253136c43fb9912ee92018555b4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 36651 } }
I20260929 11:01:35.845120 7418 raft_consensus.cc:388] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 11:01:35.845180 7418 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 11:01:35.845225 7418 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4 [term 1 FOLLOWER]: Advancing to term 2
I20260929 11:01:35.846292 7418 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78c57253136c43fb9912ee92018555b4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 36651 } }
I20260929 11:01:35.846374 7418 leader_election.cc:302] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 78c57253136c43fb9912ee92018555b4; no voters:
I20260929 11:01:35.846588 7418 leader_election.cc:288] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 11:01:35.846622 7422 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 11:01:35.846724 7422 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4 [term 2 LEADER]: Becoming Leader. State: Replica: 78c57253136c43fb9912ee92018555b4, State: Running, Role: LEADER
I20260929 11:01:35.846843 7418 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:01:35.846812 7422 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78c57253136c43fb9912ee92018555b4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 36651 } }
I20260929 11:01:35.847257 7422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "78c57253136c43fb9912ee92018555b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78c57253136c43fb9912ee92018555b4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 36651 } } }
I20260929 11:01:35.847335 7422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4 [sys.catalog]: This master's current role is: LEADER
I20260929 11:01:35.847457 7422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78c57253136c43fb9912ee92018555b4. Latest consensus state: current_term: 2 leader_uuid: "78c57253136c43fb9912ee92018555b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78c57253136c43fb9912ee92018555b4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 36651 } } }
I20260929 11:01:35.847514 7422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4 [sys.catalog]: This master's current role is: LEADER
I20260929 11:01:35.847896 7428 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:01:35.848488 7428 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:01:35.848654 7428 catalog_manager.cc:1259] Loaded cluster ID: e8de7e20562545218fb46906ec218742
I20260929 11:01:35.848724 7428 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 11:01:35.848874 7418 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260929 11:01:35.849994 7428 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:01:35.850203 7428 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4: Loaded TSK: 0
I20260929 11:01:35.850430 7428 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 11:01:35.951725 7420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:01:35.952301 7420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:01:35.952441 7420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:01:35.954128 7420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:01:35.954185 7420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:01:35.954224 7420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.193
I20260929 11:01:35.956293 7420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:43793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.26.120.212:60499/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.193:42437
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.120.193
--webserver_port=42839
--tserver_master_addrs=127.26.120.254:36651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.193
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:01:35.956671 7420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:01:35.956970 7420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:01:35.963119 7458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:01:35.963138 7456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:01:35.963445 7455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:01:35.964529 7420 server_base.cc:1034] running on GCE node
I20260929 11:01:35.964736 7420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:01:35.964999 7420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:01:35.966146 7420 hybrid_clock.cc:648] HybridClock initialized: now 1790679695966129 us; error 29 us; skew 500 ppm
Sep 29 11:01:35 dist-test-slave-2lzq krb5kdc[6386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679695, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:01:35.972641 7420 init.cc:377] Logged in from keytab as kudu/127.26.120.193@KRBTEST.COM (short username kudu)
I20260929 11:01:35.973201 7420 webserver.cc:460] Webserver started at http://127.26.120.193:42839/ using document root <none> and password file <none>
I20260929 11:01:35.973516 7420 fs_manager.cc:359] Metadata directory not provided
I20260929 11:01:35.973650 7420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 11:01:35 dist-test-slave-2lzq krb5kdc[6386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679695, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for HTTP/127.26.120.212@KRBTEST.COM
I20260929 11:01:35.990325 7420 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260929 11:01:35.991552 7466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:35.991827 7420 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:01:35.991927 7420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-0/wal
uuid: "1a66282683364798a21e557afdedd9e0"
format_stamp: "Formatted at 2026-09-29 11:01:34 on dist-test-slave-2lzq"
server_key: "6855a139b8a4cbaa1bd90f554c3447cc"
server_key_iv: "15e833791a6aed400e15d9706b9ec47a"
server_key_version: "kuduclusterkey@0"
I20260929 11:01:35.992405 7420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:01:36.013501 7420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:01:36.016094 7420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:01:36.016301 7420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:01:36.016582 7420 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:01:36.016979 7420 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:01:36.017059 7420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:36.017123 7420 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:01:36.017156 7420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:36.026537 7420 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.193:42437
I20260929 11:01:36.027161 7420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
I20260929 11:01:36.030519 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 7420
I20260929 11:01:36.030819 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.120.212:60499/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.194:33827
--local_ip_for_outbound_sockets=127.26.120.194
--tserver_master_addrs=127.26.120.254:36651
--webserver_port=33339
--webserver_interface=127.26.120.194
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:43793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.26.120.212:60499/kms with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 11:01:36.031414 7579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.193:42437 every 8 connection(s)
Sep 29 11:01:36 dist-test-slave-2lzq krb5kdc[6386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679695, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:01:36.049968 7582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:36.030502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:36651 (local address 127.26.120.193:60737)
0929 11:01:36.031712 (+ 1210us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:36.031751 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:36.032363 (+ 612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:36.033067 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:36.033071 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:36.033284 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:01:36.033534 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:36.033538 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:36.035170 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:36.035171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:36.036376 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:36.036378 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:36.037333 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:36.037334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:36.037491 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:36.038151 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:01:36.038162 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:01:36.041058 (+ 2896us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:01:36.048042 (+ 6984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:36.048051 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:36.048054 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:36.048434 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:36.048732 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:36.048733 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:36.048734 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:36.048858 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:36.049313 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:01:36.049319 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:01:36.049473 (+ 154us) client_negotiation.cc:769] Sending connection context
0929 11:01:36.049559 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 11:01:36.049719 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1167,"thread_start_us":78,"threads_started":1}
I20260929 11:01:36.051044 7583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:36.030868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:60737 (local address 127.26.120.254:36651)
0929 11:01:36.032558 (+ 1690us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:36.032567 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:36.032599 (+ 32us) server_negotiation.cc:407] Connection header received
0929 11:01:36.032732 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:36.032737 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:36.032798 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:36.032911 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:01:36.033649 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:36.035054 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:36.036470 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:36.037105 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:36.037201 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:36.041174 (+ 3973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:01:36.041178 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:01:36.041179 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:01:36.041190 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:01:36.047888 (+ 6698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:36.048546 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:36.048550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:36.048550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:36.048657 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:36.048930 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:36.048931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:36.048931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:36.049222 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:01:36.050624 (+ 1402us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:36.050701 (+ 77us) server_negotiation.cc:299] Negotiation successful
0929 11:01:36.050828 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1637,"thread_start_us":79,"threads_started":1}
I20260929 11:01:36.062374 7580 heartbeater.cc:347] Connected to a master server at 127.26.120.254:36651
I20260929 11:01:36.062497 7580 heartbeater.cc:464] Registering TS with master...
I20260929 11:01:36.062721 7580 heartbeater.cc:511] Master 127.26.120.254:36651 requested a full tablet report, sending...
W20260929 11:01:36.063664 7580 heartbeater.cc:644] Failed to heartbeat to 127.26.120.254:36651 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 11:01:36.063730 7580 heartbeater.cc:464] Registering TS with master...
I20260929 11:01:36.063810 7580 heartbeater.cc:511] Master 127.26.120.254:36651 requested a full tablet report, sending...
I20260929 11:01:36.064093 7580 heartbeater.cc:464] Registering TS with master...
I20260929 11:01:36.064155 7580 heartbeater.cc:511] Master 127.26.120.254:36651 requested a full tablet report, sending...
W20260929 11:01:36.064414 7580 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 11:01:36.161453 7584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:01:36.161818 7584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:01:36.161906 7584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:01:36.163661 7584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:01:36.163734 7584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:01:36.163781 7584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.194
I20260929 11:01:36.165853 7584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:43793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.26.120.212:60499/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.194:33827
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.120.194
--webserver_port=33339
--tserver_master_addrs=127.26.120.254:36651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.194
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:01:36.166177 7584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:01:36.166473 7584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:01:36.173043 7590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:01:36.173020 7589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:01:36.173411 7584 server_base.cc:1034] running on GCE node
W20260929 11:01:36.173103 7592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:01:36.173870 7584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:01:36.174273 7584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:01:36.175460 7584 hybrid_clock.cc:648] HybridClock initialized: now 1790679696175423 us; error 49 us; skew 500 ppm
Sep 29 11:01:36 dist-test-slave-2lzq krb5kdc[6386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679696, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:01:36.178787 7584 init.cc:377] Logged in from keytab as kudu/127.26.120.194@KRBTEST.COM (short username kudu)
I20260929 11:01:36.179382 7584 webserver.cc:460] Webserver started at http://127.26.120.194:33339/ using document root <none> and password file <none>
I20260929 11:01:36.179642 7584 fs_manager.cc:359] Metadata directory not provided
I20260929 11:01:36.179713 7584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 11:01:36 dist-test-slave-2lzq krb5kdc[6386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679696, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for HTTP/127.26.120.212@KRBTEST.COM
I20260929 11:01:36.195988 7584 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260929 11:01:36.202534 7600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:36.202803 7584 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.000s sys 0.000s
I20260929 11:01:36.202970 7584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-1/wal
uuid: "a3254e2647ae4f26b657a1b684a252e1"
format_stamp: "Formatted at 2026-09-29 11:01:35 on dist-test-slave-2lzq"
server_key: "7c97f8ac7fdc8e95b2932bc66b0926a4"
server_key_iv: "1a6ab514024eea3716635225c0b8670a"
server_key_version: "kuduclusterkey@0"
I20260929 11:01:36.203516 7584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:01:36.225934 7584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:01:36.228398 7584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:01:36.228575 7584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:01:36.228813 7584 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:01:36.229264 7584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:01:36.229316 7584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:36.229349 7584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:01:36.229404 7584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:36.237097 7584 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.194:33827
I20260929 11:01:36.237630 7584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
I20260929 11:01:36.241071 7713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.194:33827 every 8 connection(s)
I20260929 11:01:36.244313 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 7584
I20260929 11:01:36.244628 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.120.212:60499/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.195:46027
--local_ip_for_outbound_sockets=127.26.120.195
--tserver_master_addrs=127.26.120.254:36651
--webserver_port=44713
--webserver_interface=127.26.120.195
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:43793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.26.120.212:60499/kms with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 11:01:36 dist-test-slave-2lzq krb5kdc[6386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679696, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:01:36.249215 7583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:36.239834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:58629 (local address 127.26.120.254:36651)
0929 11:01:36.239923 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:36.239925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:36.239937 (+ 12us) server_negotiation.cc:407] Connection header received
0929 11:01:36.239969 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:36.239970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:36.239984 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:36.240054 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:01:36.240831 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:36.241620 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:36.242720 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:36.243049 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:36.243128 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:36.245669 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:01:36.245671 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:01:36.245671 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:01:36.245680 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:01:36.247509 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:36.248053 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:36.248054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:36.248054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:36.248121 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:36.248432 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:36.248433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:36.248433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:36.248584 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:01:36.248662 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:36.249014 (+ 352us) server_negotiation.cc:299] Negotiation successful
0929 11:01:36.249111 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 11:01:36.249464 7716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:36.238465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:36651 (local address 127.26.120.194:58629)
0929 11:01:36.238631 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:36.238679 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:36.239266 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:36.240167 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:36.240172 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:36.240419 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:01:36.240711 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:36.240718 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:36.241742 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:36.241744 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:36.242629 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:36.242631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:36.243158 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:36.243159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:36.243311 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:36.243901 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:01:36.243909 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:01:36.245559 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:01:36.247637 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:36.247644 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:36.247646 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:36.247969 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:36.248196 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:36.248197 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:36.248198 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:36.248348 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:36.248672 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:01:36.248679 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:01:36.248859 (+ 180us) client_negotiation.cc:769] Sending connection context
0929 11:01:36.248989 (+ 130us) client_negotiation.cc:240] Negotiation successful
0929 11:01:36.249166 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":85,"threads_started":1}
I20260929 11:01:36.250029 7714 heartbeater.cc:347] Connected to a master server at 127.26.120.254:36651
I20260929 11:01:36.250159 7714 heartbeater.cc:464] Registering TS with master...
I20260929 11:01:36.250375 7714 heartbeater.cc:511] Master 127.26.120.254:36651 requested a full tablet report, sending...
W20260929 11:01:36.251180 7714 heartbeater.cc:644] Failed to heartbeat to 127.26.120.254:36651 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 11:01:36.251261 7714 heartbeater.cc:464] Registering TS with master...
I20260929 11:01:36.251338 7714 heartbeater.cc:511] Master 127.26.120.254:36651 requested a full tablet report, sending...
I20260929 11:01:36.251737 7714 heartbeater.cc:464] Registering TS with master...
I20260929 11:01:36.251816 7714 heartbeater.cc:511] Master 127.26.120.254:36651 requested a full tablet report, sending...
W20260929 11:01:36.252166 7714 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 11:01:36.345304 7717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:01:36.345602 7717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:01:36.345649 7717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:01:36.347419 7717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:01:36.347556 7717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:01:36.347648 7717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.195
I20260929 11:01:36.349952 7717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:43793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.26.120.212:60499/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.195:46027
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.120.195
--webserver_port=44713
--tserver_master_addrs=127.26.120.254:36651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.195
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:01:36.350517 7717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:01:36.350886 7717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:01:36.357086 7726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:01:36.357199 7723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:01:36.357098 7724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:01:36.357496 7717 server_base.cc:1034] running on GCE node
I20260929 11:01:36.357728 7717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:01:36.357995 7717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:01:36.359198 7717 hybrid_clock.cc:648] HybridClock initialized: now 1790679696359188 us; error 28 us; skew 500 ppm
Sep 29 11:01:36 dist-test-slave-2lzq krb5kdc[6386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679696, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:01:36.361922 7717 init.cc:377] Logged in from keytab as kudu/127.26.120.195@KRBTEST.COM (short username kudu)
I20260929 11:01:36.362495 7717 webserver.cc:460] Webserver started at http://127.26.120.195:44713/ using document root <none> and password file <none>
I20260929 11:01:36.362807 7717 fs_manager.cc:359] Metadata directory not provided
I20260929 11:01:36.362937 7717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 11:01:36 dist-test-slave-2lzq krb5kdc[6386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679696, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for HTTP/127.26.120.212@KRBTEST.COM
I20260929 11:01:36.388165 7717 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:01:36.389479 7734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:36.389678 7717 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 11:01:36.389794 7717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-2/wal
uuid: "fb78903cc5f54fb98319ef03c29cde32"
format_stamp: "Formatted at 2026-09-29 11:01:35 on dist-test-slave-2lzq"
server_key: "151f153cb3643b846aa00dee49ca387c"
server_key_iv: "fd7d9bfc7e3afb83f0a656d6ff5853a2"
server_key_version: "kuduclusterkey@0"
I20260929 11:01:36.390239 7717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:01:36.413079 7717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:01:36.415501 7717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:01:36.415680 7717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:01:36.415969 7717 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:01:36.416373 7717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:01:36.416427 7717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:36.416471 7717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:01:36.416496 7717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:36.423574 7717 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.195:46027
I20260929 11:01:36.423658 7847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.195:46027 every 8 connection(s)
I20260929 11:01:36.424041 7717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
I20260929 11:01:36.424659 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 7717
Sep 29 11:01:36 dist-test-slave-2lzq krb5kdc[6386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679696, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:01:36.439458 7583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:36.424939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:47593 (local address 127.26.120.254:36651)
0929 11:01:36.424989 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:36.424991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:36.426458 (+ 1467us) server_negotiation.cc:407] Connection header received
0929 11:01:36.427065 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:36.427066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:36.427078 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:36.427150 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:01:36.428686 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:36.429436 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:36.430960 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:36.431252 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:36.431301 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:36.434813 (+ 3512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:01:36.434814 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:01:36.434815 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:01:36.434823 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:01:36.436531 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:36.437644 (+ 1113us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:36.437645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:36.437645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:36.437700 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:36.438740 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:36.438741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:36.438741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:36.438872 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:01:36.438924 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:36.439283 (+ 359us) server_negotiation.cc:299] Negotiation successful
0929 11:01:36.439366 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 11:01:36.439675 7852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:36.425262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:36651 (local address 127.26.120.195:47593)
0929 11:01:36.425587 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:36.425636 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:36.426937 (+ 1301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:36.428075 (+ 1138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:36.428081 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:36.428301 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:01:36.428572 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:36.428579 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:36.429556 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:36.429557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:36.430853 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:36.430856 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:36.431420 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:36.431421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:36.431561 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:36.432213 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:01:36.432224 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:01:36.434696 (+ 2472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:01:36.436878 (+ 2182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:36.436885 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:36.436887 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:36.437187 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:36.437784 (+ 597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:36.437785 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:36.437785 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:36.437920 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:36.438958 (+ 1038us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:01:36.438964 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:01:36.439132 (+ 168us) client_negotiation.cc:769] Sending connection context
0929 11:01:36.439246 (+ 114us) client_negotiation.cc:240] Negotiation successful
0929 11:01:36.439403 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":76,"threads_started":1}
I20260929 11:01:36.440161 7848 heartbeater.cc:347] Connected to a master server at 127.26.120.254:36651
I20260929 11:01:36.440263 7848 heartbeater.cc:464] Registering TS with master...
I20260929 11:01:36.440452 7848 heartbeater.cc:511] Master 127.26.120.254:36651 requested a full tablet report, sending...
W20260929 11:01:36.441084 7848 heartbeater.cc:644] Failed to heartbeat to 127.26.120.254:36651 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 11:01:36.441169 7848 heartbeater.cc:464] Registering TS with master...
I20260929 11:01:36.441252 7848 heartbeater.cc:511] Master 127.26.120.254:36651 requested a full tablet report, sending...
I20260929 11:01:36.441547 7848 heartbeater.cc:464] Registering TS with master...
I20260929 11:01:36.441623 7848 heartbeater.cc:511] Master 127.26.120.254:36651 requested a full tablet report, sending...
W20260929 11:01:36.441960 7848 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 11:01:36.442296 7850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:36.425060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34984 (local address 127.26.120.254:36651)
0929 11:01:36.425548 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:36.425550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:36.425560 (+ 10us) server_negotiation.cc:407] Connection header received
0929 11:01:36.425589 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:36.425590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:36.425603 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:36.425672 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:01:36.426346 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:36.427102 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:36.431029 (+ 3927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:36.433059 (+ 2030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:36.433154 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:36.436623 (+ 3469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:01:36.436624 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:01:36.436625 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:01:36.436631 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:01:36.438632 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:36.439884 (+ 1252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:36.439886 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:36.439886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:36.439952 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:36.441739 (+ 1787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:36.441740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:36.441740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:36.441898 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:01:36.442082 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:36.442143 (+ 61us) server_negotiation.cc:299] Negotiation successful
0929 11:01:36.442228 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":69,"threads_started":1}
Sep 29 11:01:36 dist-test-slave-2lzq krb5kdc[6386](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790679696, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:01:37.064597 7580 heartbeater.cc:464] Registering TS with master...
I20260929 11:01:37.064767 7580 heartbeater.cc:511] Master 127.26.120.254:36651 requested a full tablet report, sending...
I20260929 11:01:37.065279 7382 ts_manager.cc:194] Registered new tserver with Master: 1a66282683364798a21e557afdedd9e0 (127.26.120.193:42437)
I20260929 11:01:37.066335 7382 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.193@KRBTEST.COM'} at 127.26.120.193:60737
I20260929 11:01:37.252324 7714 heartbeater.cc:464] Registering TS with master...
I20260929 11:01:37.252496 7714 heartbeater.cc:511] Master 127.26.120.254:36651 requested a full tablet report, sending...
I20260929 11:01:37.252890 7382 ts_manager.cc:194] Registered new tserver with Master: a3254e2647ae4f26b657a1b684a252e1 (127.26.120.194:33827)
I20260929 11:01:37.253429 7382 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.194@KRBTEST.COM'} at 127.26.120.194:58629
Sep 29 11:01:37 dist-test-slave-2lzq krb5kdc[6386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679696, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for HTTP/127.26.120.212@KRBTEST.COM
I20260929 11:01:37.442235 7848 heartbeater.cc:464] Registering TS with master...
I20260929 11:01:37.442399 7848 heartbeater.cc:511] Master 127.26.120.254:36651 requested a full tablet report, sending...
I20260929 11:01:37.442734 7382 ts_manager.cc:194] Registered new tserver with Master: fb78903cc5f54fb98319ef03c29cde32 (127.26.120.195:46027)
I20260929 11:01:37.443248 7382 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.195@KRBTEST.COM'} at 127.26.120.195:47593
I20260929 11:01:37.452169 27107 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 11:01:37.461297 7869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:37.453186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34998 (local address 127.26.120.254:36651)
0929 11:01:37.453403 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:37.453406 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:37.453514 (+ 108us) server_negotiation.cc:407] Connection header received
0929 11:01:37.453614 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:37.453615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:37.453628 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:37.453673 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:01:37.454483 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:37.455253 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:37.456291 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:37.456601 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:37.456646 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:37.458186 (+ 1540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:01:37.458187 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:01:37.458188 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:01:37.458195 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:01:37.459986 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:37.460384 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:37.460385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:37.460386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:37.460443 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:37.460657 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:37.460659 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:37.460659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:37.460791 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:01:37.460871 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:37.461105 (+ 234us) server_negotiation.cc:299] Negotiation successful
0929 11:01:37.461192 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":60,"threads_started":1}
I20260929 11:01:37.463950 7382 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34998:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 11:01:37.466295 7382 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 11:01:37.483064 7877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:37.475926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:46027 (local address 127.0.0.1:55318)
0929 11:01:37.478194 (+ 2268us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:37.478237 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:37.478304 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:37.478792 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:37.478795 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:37.478797 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:01:37.478889 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:37.478891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:37.480207 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:37.480208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:37.481503 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:37.481506 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:37.482733 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:37.482734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:37.482847 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:37.482861 (+ 14us) client_negotiation.cc:769] Sending connection context
0929 11:01:37.482926 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 11:01:37.482978 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2236,"thread_start_us":73,"threads_started":1}
I20260929 11:01:37.483677 7878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:37.476412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55318 (local address 127.26.120.195:46027)
0929 11:01:37.478562 (+ 2150us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:37.478567 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:37.478577 (+ 10us) server_negotiation.cc:407] Connection header received
0929 11:01:37.478611 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:37.478613 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:37.478628 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:37.478701 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:01:37.478973 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:37.480108 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:37.481725 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:37.482560 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:37.482634 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:37.482688 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:37.483535 (+ 847us) server_negotiation.cc:299] Negotiation successful
0929 11:01:37.483589 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2118,"thread_start_us":58,"threads_started":1}
I20260929 11:01:37.484516 7778 tablet_service.cc:1467] Processing CreateTablet for tablet 66a8f32f409e4083a751c31440850159 (DEFAULT_TABLE table=test-table [id=fdca562497ef4c03951c20e44cc7f7e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:01:37.484895 7778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66a8f32f409e4083a751c31440850159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:01:37.491580 7876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:37.476144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:33827 (local address 127.0.0.1:41604)
0929 11:01:37.478418 (+ 2274us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:37.478426 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:37.478463 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:37.482394 (+ 3931us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:37.482395 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:37.482397 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:01:37.482452 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:37.482453 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:37.489118 (+ 6665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:37.489119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:37.490334 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:37.490336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:37.491345 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:37.491346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:37.491424 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:37.491430 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 11:01:37.491460 (+ 30us) client_negotiation.cc:240] Negotiation successful
0929 11:01:37.491502 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2254,"thread_start_us":62,"threads_started":1}
I20260929 11:01:37.492019 7879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:37.476701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41604 (local address 127.26.120.194:33827)
0929 11:01:37.482138 (+ 5437us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:37.482144 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:37.482156 (+ 12us) server_negotiation.cc:407] Connection header received
0929 11:01:37.482192 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:37.482194 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:37.482211 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:37.482299 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:01:37.482527 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:37.483580 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:37.490443 (+ 6863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:37.491238 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:37.491782 (+ 544us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:37.491866 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:37.491894 (+ 28us) server_negotiation.cc:299] Negotiation successful
0929 11:01:37.491942 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5399,"thread_start_us":58,"threads_started":1}
I20260929 11:01:37.493512 7875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:37.476288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:42437 (local address 127.0.0.1:51256)
0929 11:01:37.481616 (+ 5328us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:37.481625 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:37.481662 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:37.489437 (+ 7775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:37.489438 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:37.489441 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:01:37.489487 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:37.489489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:37.490825 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:37.490827 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:37.492290 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:37.492293 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:37.493310 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:37.493311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:37.493367 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:37.493371 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 11:01:37.493401 (+ 30us) client_negotiation.cc:240] Negotiation successful
0929 11:01:37.493441 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5312,"thread_start_us":57,"threads_started":1}
I20260929 11:01:37.494130 7880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:37.476856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51256 (local address 127.26.120.193:42437)
0929 11:01:37.489204 (+ 12348us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:37.489209 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:37.489220 (+ 11us) server_negotiation.cc:407] Connection header received
0929 11:01:37.489254 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:37.489256 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:37.489273 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:37.489346 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:01:37.489563 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:37.490715 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:37.492396 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:37.493211 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:37.493862 (+ 651us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:37.493925 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:37.493950 (+ 25us) server_negotiation.cc:299] Negotiation successful
0929 11:01:37.493989 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12306,"thread_start_us":67,"threads_started":1}
I20260929 11:01:37.494840 7648 tablet_service.cc:1467] Processing CreateTablet for tablet 66a8f32f409e4083a751c31440850159 (DEFAULT_TABLE table=test-table [id=fdca562497ef4c03951c20e44cc7f7e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:01:37.495206 7648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66a8f32f409e4083a751c31440850159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:01:37.499923 7514 tablet_service.cc:1467] Processing CreateTablet for tablet 66a8f32f409e4083a751c31440850159 (DEFAULT_TABLE table=test-table [id=fdca562497ef4c03951c20e44cc7f7e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:01:37.500272 7514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66a8f32f409e4083a751c31440850159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:01:37.504233 7881 tablet_bootstrap.cc:492] T 66a8f32f409e4083a751c31440850159 P fb78903cc5f54fb98319ef03c29cde32: Bootstrap starting.
I20260929 11:01:37.504993 7881 tablet_bootstrap.cc:654] T 66a8f32f409e4083a751c31440850159 P fb78903cc5f54fb98319ef03c29cde32: Neither blocks nor log segments found. Creating new log.
I20260929 11:01:37.505398 7881 log.cc:824] T 66a8f32f409e4083a751c31440850159 P fb78903cc5f54fb98319ef03c29cde32: Log is configured to *not* fsync() on all Append() calls
I20260929 11:01:37.506289 7881 tablet_bootstrap.cc:492] T 66a8f32f409e4083a751c31440850159 P fb78903cc5f54fb98319ef03c29cde32: No bootstrap required, opened a new log
I20260929 11:01:37.506357 7881 ts_tablet_manager.cc:1397] T 66a8f32f409e4083a751c31440850159 P fb78903cc5f54fb98319ef03c29cde32: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 11:01:37.509153 7881 raft_consensus.cc:348] T 66a8f32f409e4083a751c31440850159 P fb78903cc5f54fb98319ef03c29cde32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb78903cc5f54fb98319ef03c29cde32" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46027 } } peers { permanent_uuid: "a3254e2647ae4f26b657a1b684a252e1" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 33827 } } peers { permanent_uuid: "1a66282683364798a21e557afdedd9e0" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42437 } }
I20260929 11:01:37.509335 7881 raft_consensus.cc:374] T 66a8f32f409e4083a751c31440850159 P fb78903cc5f54fb98319ef03c29cde32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:01:37.509374 7881 raft_consensus.cc:729] T 66a8f32f409e4083a751c31440850159 P fb78903cc5f54fb98319ef03c29cde32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb78903cc5f54fb98319ef03c29cde32, State: Initialized, Role: FOLLOWER
I20260929 11:01:37.509851 7881 consensus_queue.cc:260] T 66a8f32f409e4083a751c31440850159 P fb78903cc5f54fb98319ef03c29cde32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb78903cc5f54fb98319ef03c29cde32" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46027 } } peers { permanent_uuid: "a3254e2647ae4f26b657a1b684a252e1" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 33827 } } peers { permanent_uuid: "1a66282683364798a21e557afdedd9e0" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42437 } }
I20260929 11:01:37.510213 7882 tablet_bootstrap.cc:492] T 66a8f32f409e4083a751c31440850159 P a3254e2647ae4f26b657a1b684a252e1: Bootstrap starting.
I20260929 11:01:37.510293 7881 ts_tablet_manager.cc:1428] T 66a8f32f409e4083a751c31440850159 P fb78903cc5f54fb98319ef03c29cde32: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260929 11:01:37.510609 7848 heartbeater.cc:503] Master 127.26.120.254:36651 was elected leader, sending a full tablet report...
I20260929 11:01:37.511144 7882 tablet_bootstrap.cc:654] T 66a8f32f409e4083a751c31440850159 P a3254e2647ae4f26b657a1b684a252e1: Neither blocks nor log segments found. Creating new log.
I20260929 11:01:37.511561 7882 log.cc:824] T 66a8f32f409e4083a751c31440850159 P a3254e2647ae4f26b657a1b684a252e1: Log is configured to *not* fsync() on all Append() calls
I20260929 11:01:37.512012 7883 tablet_bootstrap.cc:492] T 66a8f32f409e4083a751c31440850159 P 1a66282683364798a21e557afdedd9e0: Bootstrap starting.
I20260929 11:01:37.512658 7883 tablet_bootstrap.cc:654] T 66a8f32f409e4083a751c31440850159 P 1a66282683364798a21e557afdedd9e0: Neither blocks nor log segments found. Creating new log.
I20260929 11:01:37.514700 7883 log.cc:824] T 66a8f32f409e4083a751c31440850159 P 1a66282683364798a21e557afdedd9e0: Log is configured to *not* fsync() on all Append() calls
I20260929 11:01:37.515496 7882 tablet_bootstrap.cc:492] T 66a8f32f409e4083a751c31440850159 P a3254e2647ae4f26b657a1b684a252e1: No bootstrap required, opened a new log
I20260929 11:01:37.515599 7882 ts_tablet_manager.cc:1397] T 66a8f32f409e4083a751c31440850159 P a3254e2647ae4f26b657a1b684a252e1: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260929 11:01:37.517963 7882 raft_consensus.cc:348] T 66a8f32f409e4083a751c31440850159 P a3254e2647ae4f26b657a1b684a252e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb78903cc5f54fb98319ef03c29cde32" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46027 } } peers { permanent_uuid: "a3254e2647ae4f26b657a1b684a252e1" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 33827 } } peers { permanent_uuid: "1a66282683364798a21e557afdedd9e0" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42437 } }
I20260929 11:01:37.518237 7882 raft_consensus.cc:374] T 66a8f32f409e4083a751c31440850159 P a3254e2647ae4f26b657a1b684a252e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:01:37.518319 7882 raft_consensus.cc:729] T 66a8f32f409e4083a751c31440850159 P a3254e2647ae4f26b657a1b684a252e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3254e2647ae4f26b657a1b684a252e1, State: Initialized, Role: FOLLOWER
I20260929 11:01:37.518512 7882 consensus_queue.cc:260] T 66a8f32f409e4083a751c31440850159 P a3254e2647ae4f26b657a1b684a252e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb78903cc5f54fb98319ef03c29cde32" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46027 } } peers { permanent_uuid: "a3254e2647ae4f26b657a1b684a252e1" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 33827 } } peers { permanent_uuid: "1a66282683364798a21e557afdedd9e0" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42437 } }
I20260929 11:01:37.518848 7882 ts_tablet_manager.cc:1428] T 66a8f32f409e4083a751c31440850159 P a3254e2647ae4f26b657a1b684a252e1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 11:01:37.519001 7714 heartbeater.cc:503] Master 127.26.120.254:36651 was elected leader, sending a full tablet report...
I20260929 11:01:37.520901 7883 tablet_bootstrap.cc:492] T 66a8f32f409e4083a751c31440850159 P 1a66282683364798a21e557afdedd9e0: No bootstrap required, opened a new log
I20260929 11:01:37.521030 7883 ts_tablet_manager.cc:1397] T 66a8f32f409e4083a751c31440850159 P 1a66282683364798a21e557afdedd9e0: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260929 11:01:37.523485 7883 raft_consensus.cc:348] T 66a8f32f409e4083a751c31440850159 P 1a66282683364798a21e557afdedd9e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb78903cc5f54fb98319ef03c29cde32" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46027 } } peers { permanent_uuid: "a3254e2647ae4f26b657a1b684a252e1" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 33827 } } peers { permanent_uuid: "1a66282683364798a21e557afdedd9e0" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42437 } }
I20260929 11:01:37.523713 7883 raft_consensus.cc:374] T 66a8f32f409e4083a751c31440850159 P 1a66282683364798a21e557afdedd9e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:01:37.523806 7883 raft_consensus.cc:729] T 66a8f32f409e4083a751c31440850159 P 1a66282683364798a21e557afdedd9e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a66282683364798a21e557afdedd9e0, State: Initialized, Role: FOLLOWER
I20260929 11:01:37.524022 7883 consensus_queue.cc:260] T 66a8f32f409e4083a751c31440850159 P 1a66282683364798a21e557afdedd9e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb78903cc5f54fb98319ef03c29cde32" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46027 } } peers { permanent_uuid: "a3254e2647ae4f26b657a1b684a252e1" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 33827 } } peers { permanent_uuid: "1a66282683364798a21e557afdedd9e0" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42437 } }
I20260929 11:01:37.524365 7883 ts_tablet_manager.cc:1428] T 66a8f32f409e4083a751c31440850159 P 1a66282683364798a21e557afdedd9e0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 11:01:37.524521 7580 heartbeater.cc:503] Master 127.26.120.254:36651 was elected leader, sending a full tablet report...
W20260929 11:01:37.532264 7581 tablet.cc:2369] T 66a8f32f409e4083a751c31440850159 P 1a66282683364798a21e557afdedd9e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 11:01:37.579849 7885 raft_consensus.cc:482] T 66a8f32f409e4083a751c31440850159 P fb78903cc5f54fb98319ef03c29cde32 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 11:01:37.579990 7885 raft_consensus.cc:504] T 66a8f32f409e4083a751c31440850159 P fb78903cc5f54fb98319ef03c29cde32 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb78903cc5f54fb98319ef03c29cde32" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46027 } } peers { permanent_uuid: "a3254e2647ae4f26b657a1b684a252e1" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 33827 } } peers { permanent_uuid: "1a66282683364798a21e557afdedd9e0" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42437 } }
I20260929 11:01:37.580384 7885 leader_election.cc:288] T 66a8f32f409e4083a751c31440850159 P fb78903cc5f54fb98319ef03c29cde32 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3254e2647ae4f26b657a1b684a252e1 (127.26.120.194:33827), 1a66282683364798a21e557afdedd9e0 (127.26.120.193:42437)
Sep 29 11:01:37 dist-test-slave-2lzq krb5kdc[6386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679696, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for HTTP/127.26.120.212@KRBTEST.COM
I20260929 11:01:37.596616 7879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:37.581234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:41691 (local address 127.26.120.194:33827)
0929 11:01:37.581346 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:37.581349 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:37.590236 (+ 8887us) server_negotiation.cc:407] Connection header received
0929 11:01:37.590309 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:37.590311 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:37.590324 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:37.590405 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:01:37.590815 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:37.591592 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:37.594500 (+ 2908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:37.595043 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:37.595092 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:37.595099 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:37.596476 (+ 1377us) server_negotiation.cc:299] Negotiation successful
0929 11:01:37.596531 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260929 11:01:37.597112 7892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:37.580660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:33827 (local address 127.26.120.195:41691)
0929 11:01:37.590152 (+ 9492us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:37.590166 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:37.590249 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:37.590632 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:37.590634 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:37.590636 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:01:37.590718 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:37.590720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:37.591705 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:37.591706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:37.594392 (+ 2686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:37.594395 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:37.595162 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:37.595163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:37.595248 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:37.595251 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 11:01:37.596947 (+ 1696us) client_negotiation.cc:240] Negotiation successful
0929 11:01:37.597001 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9463,"thread_start_us":95,"threads_started":1}
I20260929 11:01:37.597679 7668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66a8f32f409e4083a751c31440850159" candidate_uuid: "fb78903cc5f54fb98319ef03c29cde32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3254e2647ae4f26b657a1b684a252e1" is_pre_election: true
I20260929 11:01:37.597872 7668 raft_consensus.cc:2454] T 66a8f32f409e4083a751c31440850159 P a3254e2647ae4f26b657a1b684a252e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb78903cc5f54fb98319ef03c29cde32 in term 0.
I20260929 11:01:37.598156 7736 leader_election.cc:302] T 66a8f32f409e4083a751c31440850159 P fb78903cc5f54fb98319ef03c29cde32 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3254e2647ae4f26b657a1b684a252e1, fb78903cc5f54fb98319ef03c29cde32; no voters:
I20260929 11:01:37.598335 7885 raft_consensus.cc:2781] T 66a8f32f409e4083a751c31440850159 P fb78903cc5f54fb98319ef03c29cde32 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 11:01:37.598408 7885 raft_consensus.cc:482] T 66a8f32f409e4083a751c31440850159 P fb78903cc5f54fb98319ef03c29cde32 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 11:01:37.598438 7885 raft_consensus.cc:3037] T 66a8f32f409e4083a751c31440850159 P fb78903cc5f54fb98319ef03c29cde32 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:01:37.598886 7880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:37.581300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:55573 (local address 127.26.120.193:42437)
0929 11:01:37.581398 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:37.581400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:37.590457 (+ 9057us) server_negotiation.cc:407] Connection header received
0929 11:01:37.590491 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:37.590493 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:37.590505 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:37.590575 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:01:37.590784 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:37.591486 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:37.592884 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:37.593416 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:37.593467 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:37.593476 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:37.598767 (+ 5291us) server_negotiation.cc:299] Negotiation successful
0929 11:01:37.598823 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260929 11:01:37.599424 7885 raft_consensus.cc:504] T 66a8f32f409e4083a751c31440850159 P fb78903cc5f54fb98319ef03c29cde32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb78903cc5f54fb98319ef03c29cde32" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46027 } } peers { permanent_uuid: "a3254e2647ae4f26b657a1b684a252e1" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 33827 } } peers { permanent_uuid: "1a66282683364798a21e557afdedd9e0" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42437 } }
I20260929 11:01:37.599558 7885 leader_election.cc:288] T 66a8f32f409e4083a751c31440850159 P fb78903cc5f54fb98319ef03c29cde32 [CANDIDATE]: Term 1 election: Requested vote from peers a3254e2647ae4f26b657a1b684a252e1 (127.26.120.194:33827), 1a66282683364798a21e557afdedd9e0 (127.26.120.193:42437)
I20260929 11:01:37.600008 7668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66a8f32f409e4083a751c31440850159" candidate_uuid: "fb78903cc5f54fb98319ef03c29cde32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3254e2647ae4f26b657a1b684a252e1"
I20260929 11:01:37.600087 7668 raft_consensus.cc:3037] T 66a8f32f409e4083a751c31440850159 P a3254e2647ae4f26b657a1b684a252e1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:01:37.600998 7668 raft_consensus.cc:2454] T 66a8f32f409e4083a751c31440850159 P a3254e2647ae4f26b657a1b684a252e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb78903cc5f54fb98319ef03c29cde32 in term 1.
I20260929 11:01:37.601075 7891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:37.580888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:42437 (local address 127.26.120.195:55573)
0929 11:01:37.590341 (+ 9453us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:37.590350 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:37.590390 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:37.590579 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:37.590580 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:37.590582 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:01:37.590696 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:37.590699 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:37.591581 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:37.591582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:37.592784 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:37.592787 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:37.593626 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:37.593627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:37.593795 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:37.593800 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 11:01:37.600934 (+ 7134us) client_negotiation.cc:240] Negotiation successful
0929 11:01:37.600997 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9439,"mutex_wait_us":30,"thread_start_us":73,"threads_started":1}
I20260929 11:01:37.601401 7736 leader_election.cc:302] T 66a8f32f409e4083a751c31440850159 P fb78903cc5f54fb98319ef03c29cde32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3254e2647ae4f26b657a1b684a252e1, fb78903cc5f54fb98319ef03c29cde32; no voters:
I20260929 11:01:37.601770 7534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66a8f32f409e4083a751c31440850159" candidate_uuid: "fb78903cc5f54fb98319ef03c29cde32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a66282683364798a21e557afdedd9e0" is_pre_election: true
I20260929 11:01:37.601943 7534 raft_consensus.cc:2454] T 66a8f32f409e4083a751c31440850159 P 1a66282683364798a21e557afdedd9e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb78903cc5f54fb98319ef03c29cde32 in term 0.
I20260929 11:01:37.602067 7533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66a8f32f409e4083a751c31440850159" candidate_uuid: "fb78903cc5f54fb98319ef03c29cde32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a66282683364798a21e557afdedd9e0"
I20260929 11:01:37.602161 7533 raft_consensus.cc:3037] T 66a8f32f409e4083a751c31440850159 P 1a66282683364798a21e557afdedd9e0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:01:37.603158 7533 raft_consensus.cc:2454] T 66a8f32f409e4083a751c31440850159 P 1a66282683364798a21e557afdedd9e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb78903cc5f54fb98319ef03c29cde32 in term 1.
I20260929 11:01:37.603559 7885 raft_consensus.cc:2781] T 66a8f32f409e4083a751c31440850159 P fb78903cc5f54fb98319ef03c29cde32 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:01:37.603730 7885 raft_consensus.cc:686] T 66a8f32f409e4083a751c31440850159 P fb78903cc5f54fb98319ef03c29cde32 [term 1 LEADER]: Becoming Leader. State: Replica: fb78903cc5f54fb98319ef03c29cde32, State: Running, Role: LEADER
I20260929 11:01:37.603822 7885 consensus_queue.cc:237] T 66a8f32f409e4083a751c31440850159 P fb78903cc5f54fb98319ef03c29cde32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb78903cc5f54fb98319ef03c29cde32" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46027 } } peers { permanent_uuid: "a3254e2647ae4f26b657a1b684a252e1" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 33827 } } peers { permanent_uuid: "1a66282683364798a21e557afdedd9e0" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42437 } }
I20260929 11:01:37.604728 7382 catalog_manager.cc:5495] T 66a8f32f409e4083a751c31440850159 P fb78903cc5f54fb98319ef03c29cde32 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb78903cc5f54fb98319ef03c29cde32 (127.26.120.195). New cstate: current_term: 1 leader_uuid: "fb78903cc5f54fb98319ef03c29cde32" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb78903cc5f54fb98319ef03c29cde32" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3254e2647ae4f26b657a1b684a252e1" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 33827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a66282683364798a21e557afdedd9e0" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42437 } health_report { overall_health: UNKNOWN } } }
I20260929 11:01:37.625166 7878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:37.621753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55330 (local address 127.26.120.195:46027)
0929 11:01:37.621819 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:37.621821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:37.621829 (+ 8us) server_negotiation.cc:407] Connection header received
0929 11:01:37.621870 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:37.621870 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:37.621887 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:37.621962 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 11:01:37.622162 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:37.622871 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:37.623975 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:37.624250 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:37.624315 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:37.624577 (+ 262us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 11:01:37.624712 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 11:01:37.624799 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:37.625047 (+ 248us) server_negotiation.cc:299] Negotiation successful
0929 11:01:37.625072 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 11:01:37.629405 7533 raft_consensus.cc:1261] T 66a8f32f409e4083a751c31440850159 P 1a66282683364798a21e557afdedd9e0 [term 1 FOLLOWER]: Refusing update from remote peer fb78903cc5f54fb98319ef03c29cde32: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 11:01:37.629406 7668 raft_consensus.cc:1261] T 66a8f32f409e4083a751c31440850159 P a3254e2647ae4f26b657a1b684a252e1 [term 1 FOLLOWER]: Refusing update from remote peer fb78903cc5f54fb98319ef03c29cde32: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 11:01:37.629720 7885 consensus_queue.cc:1035] T 66a8f32f409e4083a751c31440850159 P fb78903cc5f54fb98319ef03c29cde32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a66282683364798a21e557afdedd9e0" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 11:01:37.630009 7893 consensus_queue.cc:1035] T 66a8f32f409e4083a751c31440850159 P fb78903cc5f54fb98319ef03c29cde32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3254e2647ae4f26b657a1b684a252e1" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 11:01:37.636593 7897 mvcc.cc:205] Tried to move back new op lower bound from 7334624041486770176 to 7334624041385861120. Current Snapshot: MvccSnapshot[applied={T|T < 7334624041486770176}]
I20260929 11:01:37.640059 7898 mvcc.cc:205] Tried to move back new op lower bound from 7334624041486770176 to 7334624041385861120. Current Snapshot: MvccSnapshot[applied={T|T < 7334624041486770176}]
I20260929 11:01:37.642489 7899 mvcc.cc:205] Tried to move back new op lower bound from 7334624041486770176 to 7334624041385861120. Current Snapshot: MvccSnapshot[applied={T|T < 7334624041486770176}]
I20260929 11:01:37.644898 7381 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 11:01:37.645000 7381 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 11:01:37.645767 7381 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 78c57253136c43fb9912ee92018555b4: Sending DeleteTablet for 3 replicas of tablet 66a8f32f409e4083a751c31440850159
I20260929 11:01:37.646237 7778 tablet_service.cc:1514] Processing DeleteTablet for tablet 66a8f32f409e4083a751c31440850159 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:01:37 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:55318
I20260929 11:01:37.646430 7648 tablet_service.cc:1514] Processing DeleteTablet for tablet 66a8f32f409e4083a751c31440850159 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:01:37 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:41604
I20260929 11:01:37.646613 7514 tablet_service.cc:1514] Processing DeleteTablet for tablet 66a8f32f409e4083a751c31440850159 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:01:37 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:51256
I20260929 11:01:37.647331 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 7420
I20260929 11:01:37.652101 7907 tablet_replica.cc:330] T 66a8f32f409e4083a751c31440850159 P fb78903cc5f54fb98319ef03c29cde32: stopping tablet replica
I20260929 11:01:37.652240 7908 tablet_replica.cc:330] T 66a8f32f409e4083a751c31440850159 P a3254e2647ae4f26b657a1b684a252e1: stopping tablet replica
I20260929 11:01:37.653398 7908 raft_consensus.cc:2229] T 66a8f32f409e4083a751c31440850159 P a3254e2647ae4f26b657a1b684a252e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 11:01:37.653555 7907 raft_consensus.cc:2229] T 66a8f32f409e4083a751c31440850159 P fb78903cc5f54fb98319ef03c29cde32 [term 1 LEADER]: Raft consensus shutting down.
I20260929 11:01:37.653717 7907 raft_consensus.cc:2258] T 66a8f32f409e4083a751c31440850159 P fb78903cc5f54fb98319ef03c29cde32 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 11:01:37.653855 7908 raft_consensus.cc:2258] T 66a8f32f409e4083a751c31440850159 P a3254e2647ae4f26b657a1b684a252e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 11:01:37.654882 7908 ts_tablet_manager.cc:1905] T 66a8f32f409e4083a751c31440850159 P a3254e2647ae4f26b657a1b684a252e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 11:01:37.656864 7907 ts_tablet_manager.cc:1905] T 66a8f32f409e4083a751c31440850159 P fb78903cc5f54fb98319ef03c29cde32: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 11:01:37.665675 7907 ts_tablet_manager.cc:1918] T 66a8f32f409e4083a751c31440850159 P fb78903cc5f54fb98319ef03c29cde32: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 11:01:37.686857 7908 ts_tablet_manager.cc:1918] T 66a8f32f409e4083a751c31440850159 P a3254e2647ae4f26b657a1b684a252e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 11:01:37.687088 7907 log.cc:1196] T 66a8f32f409e4083a751c31440850159 P fb78903cc5f54fb98319ef03c29cde32: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-2/wal/wals/66a8f32f409e4083a751c31440850159
I20260929 11:01:37.687589 7907 ts_tablet_manager.cc:1939] T 66a8f32f409e4083a751c31440850159 P fb78903cc5f54fb98319ef03c29cde32: Deleting consensus metadata
I20260929 11:01:37.687770 7908 log.cc:1196] T 66a8f32f409e4083a751c31440850159 P a3254e2647ae4f26b657a1b684a252e1: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790679549986039-27107-0/minicluster-data/ts-1/wal/wals/66a8f32f409e4083a751c31440850159
I20260929 11:01:37.688119 7908 ts_tablet_manager.cc:1939] T 66a8f32f409e4083a751c31440850159 P a3254e2647ae4f26b657a1b684a252e1: Deleting consensus metadata
I20260929 11:01:37.688632 7367 catalog_manager.cc:4841] TS fb78903cc5f54fb98319ef03c29cde32 (127.26.120.195:46027): tablet 66a8f32f409e4083a751c31440850159 (table test-table [id=fdca562497ef4c03951c20e44cc7f7e1]) successfully deleted
I20260929 11:01:37.689167 7367 catalog_manager.cc:4841] TS a3254e2647ae4f26b657a1b684a252e1 (127.26.120.194:33827): tablet 66a8f32f409e4083a751c31440850159 (table test-table [id=fdca562497ef4c03951c20e44cc7f7e1]) successfully deleted
W20260929 11:01:37.689427 7367 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.120.193:42437 (error 108)
I20260929 11:01:37.690580 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 7584
I20260929 11:01:37.697510 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 7717
I20260929 11:01:37.702548 7875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:37.702402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:42437 (local address 127.0.0.1:51268)
0929 11:01:37.702460 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:37.702502 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.193:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260929 11:01:37.702749 7367 catalog_manager.cc:4584] TS 1a66282683364798a21e557afdedd9e0 (127.26.120.193:42437): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 66a8f32f409e4083a751c31440850159: Network error: Client connection negotiation failed: client connection to 127.26.120.193:42437: connect: Connection refused (error 111)
I20260929 11:01:37.703495 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 7349
I20260929 11:01:37.709491 27107 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260929 11:01:38.053917 27107 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260929 11:01:38.054006 27107 mini_ranger.cc:66] Stopping Ranger...
I20260929 11:01:38.398818 27107 mini_ranger.cc:68] Stopped Ranger
2026-09-29 11:01:38.398 UTC [6416] LOG: received smart shutdown request
2026-09-29 11:01:38.403 UTC [6416] LOG: background worker "logical replication launcher" (PID 6425) exited with exit code 1
2026-09-29 11:01:38.403 UTC [6420] LOG: shutting down
2026-09-29 11:01:38.579 UTC [6416] LOG: database system is shut down
2026-09-29T11:01:38Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (48129 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 11:01:38 dist-test-slave-2lzq krb5kdc[7919](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 11:01:38 dist-test-slave-2lzq krb5kdc[7919](info): set up 2 sockets
Sep 29 11:01:38 dist-test-slave-2lzq krb5kdc[7919](info): commencing operation
krb5kdc: starting...
W20260929 11:01:39.714409 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.003s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 11:01:39 dist-test-slave-2lzq krb5kdc[7919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T11:01:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T11:01:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.120.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
I20260929 11:01:39.831104 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:43835
--webserver_interface=127.26.120.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:40947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:43835,127.26.120.253:36357,127.26.120.252:45869
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:01:39.910662 7932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:01:39.910866 7932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:01:39.910895 7932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:01:39.911960 7932 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:01:39.911993 7932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:01:39.912006 7932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:01:39.912019 7932 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:01:39.912031 7932 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:01:39.913410 7932 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:40947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:43835,127.26.120.253:36357,127.26.120.252:45869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:43835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:01:39.913601 7932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:01:39.913769 7932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:01:39.918483 7940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:01:39.918512 7937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:01:39.918483 7938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:01:39.918507 7932 server_base.cc:1034] running on GCE node
I20260929 11:01:39.918859 7932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:01:39.919157 7932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:01:39.920341 7932 hybrid_clock.cc:648] HybridClock initialized: now 1790679699920321 us; error 35 us; skew 500 ppm
Sep 29 11:01:39 dist-test-slave-2lzq krb5kdc[7919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679699, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:01:39.928125 7932 init.cc:377] Logged in from keytab as kudu/127.26.120.254@KRBTEST.COM (short username kudu)
I20260929 11:01:39.928591 7932 webserver.cc:460] Webserver started at http://127.26.120.254:37065/ using document root <none> and password file <none>
I20260929 11:01:39.928817 7932 fs_manager.cc:359] Metadata directory not provided
I20260929 11:01:39.928886 7932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:01:39.929035 7932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:01:39.930150 7932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "4538f5ba711a46b6854bd148ec45cb64"
format_stamp: "Formatted at 2026-09-29 11:01:39 on dist-test-slave-2lzq"
server_key: "4e5513b676c1d81621178f72543d7d85"
server_key_iv: "02dda6b550a50ebe84554c5e3ff85793"
server_key_version: "encryptionkey@0"
I20260929 11:01:39.930531 7932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "4538f5ba711a46b6854bd148ec45cb64"
format_stamp: "Formatted at 2026-09-29 11:01:39 on dist-test-slave-2lzq"
server_key: "4e5513b676c1d81621178f72543d7d85"
server_key_iv: "02dda6b550a50ebe84554c5e3ff85793"
server_key_version: "encryptionkey@0"
I20260929 11:01:39.931842 7932 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 11:01:39.932540 7947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:39.932711 7932 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 11:01:39.932796 7932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "4538f5ba711a46b6854bd148ec45cb64"
format_stamp: "Formatted at 2026-09-29 11:01:39 on dist-test-slave-2lzq"
server_key: "4e5513b676c1d81621178f72543d7d85"
server_key_iv: "02dda6b550a50ebe84554c5e3ff85793"
server_key_version: "encryptionkey@0"
I20260929 11:01:39.932885 7932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:01:39.939661 7932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:01:39.941442 7932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:01:39.941592 7932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:01:39.945403 7932 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:43835
I20260929 11:01:39.945497 8000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:43835 every 8 connection(s)
I20260929 11:01:39.945799 7932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:01:39.946660 8001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:01:39.948107 8001 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43835 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 11:01:39 dist-test-slave-2lzq krb5kdc[7919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679699, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:01:39.955785 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 7932
I20260929 11:01:39.955924 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 11:01:39.956100 27107 external_mini_cluster.cc:1428] Setting key 647f399c5cebf23c0b3da5587e1757af
I20260929 11:01:39.957931 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:39.948711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43835 (local address 127.0.0.1:55806)
0929 11:01:39.948903 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:39.948944 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:39.949428 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:39.949741 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:39.949743 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:39.950013 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:01:39.950213 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:39.950218 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:39.951242 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:39.951243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:39.951925 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:39.951927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:39.952440 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:39.952441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:39.952504 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:39.953009 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:01:39.953018 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:01:39.954379 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:01:39.956222 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:39.956230 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:39.956233 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:39.956496 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:39.956772 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:39.956773 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:39.956773 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:39.956886 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:39.957262 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:01:39.957269 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:01:39.957360 (+ 91us) client_negotiation.cc:769] Sending connection context
0929 11:01:39.957462 (+ 102us) client_negotiation.cc:240] Negotiation successful
0929 11:01:39.957777 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":122,"thread_start_us":95,"threads_started":1}
I20260929 11:01:39.957940 8003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:39.948731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55806 (local address 127.26.120.254:43835)
0929 11:01:39.948911 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:39.948920 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:39.949057 (+ 137us) server_negotiation.cc:407] Connection header received
0929 11:01:39.949621 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:39.949623 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:39.949639 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:39.949736 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:01:39.950309 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:39.951128 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:39.952034 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:39.952342 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:39.952466 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:39.954467 (+ 2001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:01:39.954474 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:01:39.954474 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:01:39.954480 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:01:39.956134 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:39.956611 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:39.956614 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:39.956615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:39.956696 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:39.956951 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:39.956952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:39.956952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:39.957183 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:01:39.957255 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:39.957496 (+ 241us) server_negotiation.cc:299] Negotiation successful
0929 11:01:39.957676 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":86,"threads_started":1}
I20260929 11:01:39.958839 8001 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.120.253" port: 36357 } has no permanent_uuid. Determining permanent_uuid...
I20260929 11:01:39.959251 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:39.959093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.253:36357 (local address 127.0.0.1:46700)
0929 11:01:39.959145 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:39.959219 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.253:36357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260929 11:01:39.959385 7949 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.120.253:36357: connect: Connection refused (error 111)
I20260929 11:01:39.959820 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:39.959709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.253:36357 (local address 127.0.0.1:46714)
0929 11:01:39.959760 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:39.959797 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.253:36357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260929 11:01:39.959949 8001 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.120.253:36357: Network error: Client connection negotiation failed: client connection to 127.26.120.253:36357: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.26.120.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.253.keytab.
Entry for principal kudu/127.26.120.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.253.keytab.
I20260929 11:01:39.981400 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.253:36357
--webserver_interface=127.26.120.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.253.keytab
--principal=kudu/127.26.120.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:40947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:43835,127.26.120.253:36357,127.26.120.252:45869
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 11:01:40.009194 8001 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.120.253" port: 36357 } attempt: 1
I20260929 11:01:40.009693 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:40.009549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.253:36357 (local address 127.0.0.1:46720)
0929 11:01:40.009631 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:40.009664 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.253:36357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260929 11:01:40.010023 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:40.009930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.253:36357 (local address 127.0.0.1:46736)
0929 11:01:40.009975 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:40.009999 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.253:36357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260929 11:01:40.010128 8001 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.120.253:36357: Network error: Client connection negotiation failed: client connection to 127.26.120.253:36357: connect: Connection refused (error 111)
W20260929 11:01:40.061668 8008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:01:40.061911 8008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:01:40.061946 8008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:01:40.063094 8008 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:01:40.063165 8008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:01:40.063189 8008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:01:40.063210 8008 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:01:40.063239 8008 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:01:40.064689 8008 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:40947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:43835,127.26.120.253:36357,127.26.120.252:45869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.253.keytab
--principal=kudu/127.26.120.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.253:36357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.120.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:01:40.064958 8008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:01:40.065186 8008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:01:40.069998 8013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:01:40.069994 8014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:01:40.070199 8016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:01:40.070698 8008 server_base.cc:1034] running on GCE node
I20260929 11:01:40.070995 8008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:01:40.071314 8008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:01:40.072484 8008 hybrid_clock.cc:648] HybridClock initialized: now 1790679700072468 us; error 26 us; skew 500 ppm
Sep 29 11:01:40 dist-test-slave-2lzq krb5kdc[7919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679700, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:01:40.075100 8008 init.cc:377] Logged in from keytab as kudu/127.26.120.253@KRBTEST.COM (short username kudu)
I20260929 11:01:40.075482 8008 webserver.cc:460] Webserver started at http://127.26.120.253:43047/ using document root <none> and password file <none>
I20260929 11:01:40.075690 8008 fs_manager.cc:359] Metadata directory not provided
I20260929 11:01:40.075731 8008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:01:40.075815 8008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:01:40.076776 8008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-1/data/instance:
uuid: "39bdbda009ed43d0956c70d48a5bff8b"
format_stamp: "Formatted at 2026-09-29 11:01:40 on dist-test-slave-2lzq"
server_key: "cfb6ee655fb5b61f36b345d4f4f31f09"
server_key_iv: "ccc4077cd448fd03c9ac36aa8be73e6a"
server_key_version: "encryptionkey@0"
I20260929 11:01:40.077087 8008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-1/wal/instance:
uuid: "39bdbda009ed43d0956c70d48a5bff8b"
format_stamp: "Formatted at 2026-09-29 11:01:40 on dist-test-slave-2lzq"
server_key: "cfb6ee655fb5b61f36b345d4f4f31f09"
server_key_iv: "ccc4077cd448fd03c9ac36aa8be73e6a"
server_key_version: "encryptionkey@0"
I20260929 11:01:40.078266 8001 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.120.253" port: 36357 } attempt: 2
I20260929 11:01:40.078400 8008 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 11:01:40.078667 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:40.078543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.253:36357 (local address 127.0.0.1:46744)
0929 11:01:40.078602 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:40.078636 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.253:36357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260929 11:01:40.079070 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:40.078957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.253:36357 (local address 127.0.0.1:46750)
0929 11:01:40.079000 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:40.079046 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.253:36357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260929 11:01:40.079228 8001 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.120.253:36357: Network error: Client connection negotiation failed: client connection to 127.26.120.253:36357: connect: Connection refused (error 111)
I20260929 11:01:40.079310 8023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:40.079445 8008 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 11:01:40.079540 8008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-1/wal
uuid: "39bdbda009ed43d0956c70d48a5bff8b"
format_stamp: "Formatted at 2026-09-29 11:01:40 on dist-test-slave-2lzq"
server_key: "cfb6ee655fb5b61f36b345d4f4f31f09"
server_key_iv: "ccc4077cd448fd03c9ac36aa8be73e6a"
server_key_version: "encryptionkey@0"
I20260929 11:01:40.079615 8008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:01:40.091619 8008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:01:40.093852 8008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:01:40.094040 8008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:01:40.098425 8008 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.253:36357
I20260929 11:01:40.098513 8075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.253:36357 every 8 connection(s)
I20260929 11:01:40.098853 8008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-1/data/info.pb
I20260929 11:01:40.099805 8076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:01:40.101329 8076 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43835 } has no permanent_uuid. Determining permanent_uuid...
I20260929 11:01:40.106196 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 8008
I20260929 11:01:40.106315 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-1/wal/instance
I20260929 11:01:40.106492 27107 external_mini_cluster.cc:1428] Setting key e59cc44f759f9c351c996ffeded93523
Sep 29 11:01:40 dist-test-slave-2lzq krb5kdc[7919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679700, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.253@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:01:40.110767 8003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:40.102002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55814 (local address 127.26.120.254:43835)
0929 11:01:40.102099 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:40.102101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:40.102336 (+ 235us) server_negotiation.cc:407] Connection header received
0929 11:01:40.102791 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:40.102792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:40.102804 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:40.102858 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:01:40.103441 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.103930 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.104948 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.105148 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.105202 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:40.107589 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:01:40.107591 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:01:40.107591 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:01:40.107598 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:01:40.109176 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:40.109651 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:40.109653 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:40.109653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:40.109721 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:40.109961 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:40.109962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:40.109963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:40.110159 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:01:40.110239 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:40.110564 (+ 325us) server_negotiation.cc:299] Negotiation successful
0929 11:01:40.110661 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 11:01:40.110797 8078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:40.101987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43835 (local address 127.0.0.1:55814)
0929 11:01:40.102198 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:40.102238 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:40.102708 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:40.102915 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:40.102918 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:40.103160 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:01:40.103372 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:40.103376 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.104016 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.104017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:40.104873 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:40.104875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.105222 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.105223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:40.105359 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:40.105831 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:01:40.105838 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:01:40.107494 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:01:40.109283 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:40.109290 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:40.109291 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:40.109574 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:40.109789 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:40.109790 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:40.109790 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:40.109897 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:40.110227 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:01:40.110230 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:01:40.110411 (+ 181us) client_negotiation.cc:769] Sending connection context
0929 11:01:40.110514 (+ 103us) client_negotiation.cc:240] Negotiation successful
0929 11:01:40.110640 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":127,"threads_started":1}
I20260929 11:01:40.111431 8076 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.120.253" port: 36357 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.26.120.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.252@KRBTEST.COM" created.
Sep 29 11:01:40 dist-test-slave-2lzq krb5kdc[7919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679700, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.253@KRBTEST.COM for kudu/127.26.120.253@KRBTEST.COM
I20260929 11:01:40.121411 8078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:40.111696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.253:36357 (local address 127.0.0.1:46754)
0929 11:01:40.111746 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:40.111758 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:40.111797 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:40.112063 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:40.112064 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:40.112234 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:01:40.112305 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:40.112307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.113154 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.113156 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:40.113989 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:40.113992 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.114566 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.114567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:40.114649 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:40.115130 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:01:40.115135 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:01:40.118357 (+ 3222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:01:40.120200 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:40.120201 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:40.120201 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:40.120393 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:40.120669 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:40.120670 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:40.120670 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:40.120729 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:40.121044 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:01:40.121045 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:01:40.121086 (+ 41us) client_negotiation.cc:769] Sending connection context
0929 11:01:40.121176 (+ 90us) client_negotiation.cc:240] Negotiation successful
0929 11:01:40.121288 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260929 11:01:40.121511 8080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:40.111773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46754 (local address 127.26.120.253:36357)
0929 11:01:40.111915 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:40.111919 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:40.111936 (+ 17us) server_negotiation.cc:407] Connection header received
0929 11:01:40.111960 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:40.111962 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:40.111972 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:40.112027 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:01:40.112389 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.113062 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.114141 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.114475 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.114539 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:40.118444 (+ 3905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:01:40.118453 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:01:40.118453 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:01:40.118460 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:01:40.120107 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:40.120534 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:40.120538 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:40.120538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:40.120599 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:40.120815 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:40.120816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:40.120816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:40.120973 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:01:40.121067 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:40.121211 (+ 144us) server_negotiation.cc:299] Negotiation successful
0929 11:01:40.121383 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":79,"server-negotiator.queue_time_us":116,"thread_start_us":92,"threads_started":1}
I20260929 11:01:40.122112 8076 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.120.252" port: 45869 } has no permanent_uuid. Determining permanent_uuid...
I20260929 11:01:40.122534 8078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:40.122406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:45869 (local address 127.0.0.1:43902)
0929 11:01:40.122460 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:40.122506 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.252:45869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260929 11:01:40.122677 8025 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.120.252:45869: connect: Connection refused (error 111)
I20260929 11:01:40.123095 8078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:40.122992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:45869 (local address 127.0.0.1:43918)
0929 11:01:40.123047 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:40.123071 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.252:45869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260929 11:01:40.123194 8076 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.120.252:45869: Network error: Client connection negotiation failed: client connection to 127.26.120.252:45869: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.252.keytab.
Entry for principal kudu/127.26.120.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.252.keytab.
I20260929 11:01:40.133678 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.252:45869
--webserver_interface=127.26.120.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.252.keytab
--principal=kudu/127.26.120.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:40947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:43835,127.26.120.253:36357,127.26.120.252:45869
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 11:01:40.170392 8001 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.120.253" port: 36357 } attempt: 3
I20260929 11:01:40.172405 8076 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.120.252" port: 45869 } attempt: 1
I20260929 11:01:40.172829 8078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:40.172692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:45869 (local address 127.0.0.1:43930)
0929 11:01:40.172752 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:40.172790 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.252:45869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260929 11:01:40.173210 8078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:40.173117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:45869 (local address 127.0.0.1:43936)
0929 11:01:40.173161 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:40.173185 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.252:45869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260929 11:01:40.173368 8076 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.120.252:45869: Network error: Client connection negotiation failed: client connection to 127.26.120.252:45869: connect: Connection refused (error 111)
Sep 29 11:01:40 dist-test-slave-2lzq krb5kdc[7919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679699, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for kudu/127.26.120.253@KRBTEST.COM
I20260929 11:01:40.178253 8080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:40.170818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46760 (local address 127.26.120.253:36357)
0929 11:01:40.170957 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:40.170959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:40.170966 (+ 7us) server_negotiation.cc:407] Connection header received
0929 11:01:40.170989 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:40.170990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:40.170999 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:40.171039 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:01:40.171461 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.171996 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.173048 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.173385 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.173446 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:40.175440 (+ 1994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:01:40.175442 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:01:40.175443 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:01:40.175450 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:01:40.176981 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:40.177369 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:40.177370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:40.177370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:40.177431 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:40.177680 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:40.177681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:40.177682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:40.177825 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:01:40.177906 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:40.178092 (+ 186us) server_negotiation.cc:299] Negotiation successful
0929 11:01:40.178167 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260929 11:01:40.178259 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:40.170748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.253:36357 (local address 127.0.0.1:46760)
0929 11:01:40.170825 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:40.170838 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:40.170889 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:40.171079 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:40.171081 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:40.171270 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:01:40.171345 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:40.171347 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.172108 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.172109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:40.172933 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:40.172936 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.173505 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.173506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:40.173583 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:40.174006 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:01:40.174011 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:01:40.175361 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:01:40.177075 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:40.177076 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:40.177076 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:40.177258 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:40.177518 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:40.177519 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:40.177519 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:40.177583 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:40.177910 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:01:40.177911 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:01:40.177961 (+ 50us) client_negotiation.cc:769] Sending connection context
0929 11:01:40.178031 (+ 70us) client_negotiation.cc:240] Negotiation successful
0929 11:01:40.178164 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260929 11:01:40.179059 8001 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.120.252" port: 45869 } has no permanent_uuid. Determining permanent_uuid...
I20260929 11:01:40.179420 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:40.179289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:45869 (local address 127.0.0.1:43944)
0929 11:01:40.179367 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:40.179398 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.252:45869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260929 11:01:40.179746 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:40.179667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:45869 (local address 127.0.0.1:43958)
0929 11:01:40.179709 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:40.179732 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.252:45869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260929 11:01:40.179853 8001 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.120.252:45869: Network error: Client connection negotiation failed: client connection to 127.26.120.252:45869: connect: Connection refused (error 111)
I20260929 11:01:40.211038 8001 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.120.252" port: 45869 } attempt: 1
I20260929 11:01:40.211653 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:40.211420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:45869 (local address 127.0.0.1:43966)
0929 11:01:40.211520 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:40.211605 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.252:45869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260929 11:01:40.212159 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:40.212024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:45869 (local address 127.0.0.1:43980)
0929 11:01:40.212100 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:40.212133 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.252:45869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260929 11:01:40.212306 8001 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.120.252:45869: Network error: Client connection negotiation failed: client connection to 127.26.120.252:45869: connect: Connection refused (error 111)
W20260929 11:01:40.218423 8083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:01:40.218628 8083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:01:40.218652 8083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:01:40.219822 8083 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:01:40.219864 8083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:01:40.219879 8083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:01:40.219892 8083 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:01:40.219904 8083 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:01:40.221398 8083 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:40947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:43835,127.26.120.253:36357,127.26.120.252:45869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.252.keytab
--principal=kudu/127.26.120.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.252:45869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.120.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:01:40.221653 8083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:01:40.221900 8083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 11:01:40.227013 8083 server_base.cc:1034] running on GCE node
W20260929 11:01:40.226963 8089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:01:40.226963 8088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:01:40.226972 8091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:01:40.227453 8083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:01:40.227787 8083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:01:40.228982 8083 hybrid_clock.cc:648] HybridClock initialized: now 1790679700228958 us; error 34 us; skew 500 ppm
Sep 29 11:01:40 dist-test-slave-2lzq krb5kdc[7919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679700, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:01:40.231472 8083 init.cc:377] Logged in from keytab as kudu/127.26.120.252@KRBTEST.COM (short username kudu)
I20260929 11:01:40.231855 8083 webserver.cc:460] Webserver started at http://127.26.120.252:35021/ using document root <none> and password file <none>
I20260929 11:01:40.232030 8083 fs_manager.cc:359] Metadata directory not provided
I20260929 11:01:40.232070 8083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:01:40.232197 8083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:01:40.233218 8083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-2/data/instance:
uuid: "c1c80174493e4a04b785197fcde474e3"
format_stamp: "Formatted at 2026-09-29 11:01:40 on dist-test-slave-2lzq"
server_key: "6a41157eac210650424ce7728ff6c318"
server_key_iv: "80d14e1d5035d1e1221316746ca095ff"
server_key_version: "encryptionkey@0"
I20260929 11:01:40.233518 8083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-2/wal/instance:
uuid: "c1c80174493e4a04b785197fcde474e3"
format_stamp: "Formatted at 2026-09-29 11:01:40 on dist-test-slave-2lzq"
server_key: "6a41157eac210650424ce7728ff6c318"
server_key_iv: "80d14e1d5035d1e1221316746ca095ff"
server_key_version: "encryptionkey@0"
I20260929 11:01:40.234856 8083 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 11:01:40.235587 8098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:40.235806 8083 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:01:40.235939 8083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-2/wal
uuid: "c1c80174493e4a04b785197fcde474e3"
format_stamp: "Formatted at 2026-09-29 11:01:40 on dist-test-slave-2lzq"
server_key: "6a41157eac210650424ce7728ff6c318"
server_key_iv: "80d14e1d5035d1e1221316746ca095ff"
server_key_version: "encryptionkey@0"
I20260929 11:01:40.236014 8083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:01:40.241516 8076 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.120.252" port: 45869 } attempt: 2
I20260929 11:01:40.241945 8078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:40.241817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:45869 (local address 127.0.0.1:43982)
0929 11:01:40.241875 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:40.241915 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.252:45869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260929 11:01:40.242274 8078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:40.242193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:45869 (local address 127.0.0.1:43998)
0929 11:01:40.242234 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:40.242258 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.252:45869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260929 11:01:40.242395 8076 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.120.252:45869: Network error: Client connection negotiation failed: client connection to 127.26.120.252:45869: connect: Connection refused (error 111)
I20260929 11:01:40.245121 8083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:01:40.247043 8083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:01:40.247200 8083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:01:40.251005 8083 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.252:45869
I20260929 11:01:40.251055 8150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.252:45869 every 8 connection(s)
I20260929 11:01:40.251386 8083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-2/data/info.pb
I20260929 11:01:40.252233 8151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:01:40.253820 8151 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43835 } has no permanent_uuid. Determining permanent_uuid...
I20260929 11:01:40.258796 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 8083
I20260929 11:01:40.258899 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790679549986039-27107-0/minicluster-data/master-2/wal/instance
I20260929 11:01:40.259076 27107 external_mini_cluster.cc:1428] Setting key 406b3f54860b2c7a6866cd58a5dce932
Sep 29 11:01:40 dist-test-slave-2lzq krb5kdc[7919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679700, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.252@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:01:40.263041 8003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:40.254538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55826 (local address 127.26.120.254:43835)
0929 11:01:40.254617 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:40.254620 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:40.254819 (+ 199us) server_negotiation.cc:407] Connection header received
0929 11:01:40.255236 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:40.255238 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:40.255246 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:40.255298 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:01:40.255852 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.256314 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.257231 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.257438 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.257494 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:40.259953 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:01:40.259954 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:01:40.259955 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:01:40.259962 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:01:40.261540 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:40.262022 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:40.262023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:40.262023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:40.262114 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:40.262386 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:40.262386 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:40.262386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:40.262488 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:01:40.262540 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:40.262853 (+ 313us) server_negotiation.cc:299] Negotiation successful
0929 11:01:40.262933 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 11:01:40.263118 8153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:40.254507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43835 (local address 127.0.0.1:55826)
0929 11:01:40.254700 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:40.254736 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:40.255158 (+ 422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:40.255347 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:40.255350 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:40.255591 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:01:40.255784 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:40.255787 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.256390 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.256391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:40.257155 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:40.257157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.257523 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.257524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:40.257625 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:40.258181 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:01:40.258189 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:01:40.259890 (+ 1701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:01:40.261621 (+ 1731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:40.261628 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:40.261629 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:40.261941 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:40.262184 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:40.262185 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:40.262185 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:40.262321 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:40.262557 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:01:40.262563 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:01:40.262729 (+ 166us) client_negotiation.cc:769] Sending connection context
0929 11:01:40.262812 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 11:01:40.262928 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":114,"threads_started":1}
I20260929 11:01:40.263916 8151 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.120.253" port: 36357 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 11:01:40 dist-test-slave-2lzq krb5kdc[7919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.252@KRBTEST.COM
Sep 29 11:01:40 dist-test-slave-2lzq krb5kdc[7919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679700, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.252@KRBTEST.COM for kudu/127.26.120.253@KRBTEST.COM
I20260929 11:01:40.268812 8155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:40.259480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44008 (local address 127.26.120.252:45869)
0929 11:01:40.259640 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:40.259645 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:40.259683 (+ 38us) server_negotiation.cc:407] Connection header received
0929 11:01:40.259725 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:40.259728 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:40.259749 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:40.259832 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:01:40.260453 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.261614 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.262692 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.263089 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.263157 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:40.265766 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:01:40.265772 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:01:40.265773 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:01:40.265779 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:01:40.267519 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:40.267911 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:40.267913 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:40.267914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:40.267976 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:40.268187 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:40.268188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:40.268188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:40.268378 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:01:40.268552 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:40.268600 (+ 48us) server_negotiation.cc:299] Negotiation successful
0929 11:01:40.268680 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":83,"threads_started":1}
I20260929 11:01:40.270642 8153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:40.264171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.253:36357 (local address 127.0.0.1:46770)
0929 11:01:40.264219 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:40.264228 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:40.264265 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:40.264440 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:40.264441 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:40.264552 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:01:40.264596 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:40.264597 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.265265 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.265266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:40.265948 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:40.265950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.266371 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.266372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:40.266446 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:40.266770 (+ 324us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:01:40.266774 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:01:40.267856 (+ 1082us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:01:40.269626 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:40.269627 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:40.269627 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:40.269844 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:40.270048 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:40.270049 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:40.270049 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:40.270135 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:40.270368 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:01:40.270369 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:01:40.270411 (+ 42us) client_negotiation.cc:769] Sending connection context
0929 11:01:40.270509 (+ 98us) client_negotiation.cc:240] Negotiation successful
0929 11:01:40.270587 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260929 11:01:40.270699 8080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:40.264266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46770 (local address 127.26.120.253:36357)
0929 11:01:40.264312 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:40.264314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:40.264324 (+ 10us) server_negotiation.cc:407] Connection header received
0929 11:01:40.264352 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:40.264352 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:40.264364 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:40.264439 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:01:40.264669 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.265186 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.266034 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.266283 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.266371 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:40.267947 (+ 1576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:01:40.267949 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:01:40.267950 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:01:40.267957 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:01:40.269542 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:40.269913 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:40.269915 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:40.269915 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:40.269983 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:40.270201 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:40.270202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:40.270202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:40.270302 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:01:40.270355 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:40.270531 (+ 176us) server_negotiation.cc:299] Negotiation successful
0929 11:01:40.270610 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 11:01:40.271121 8151 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.120.252" port: 45869 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 11:01:40 dist-test-slave-2lzq krb5kdc[7919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
Sep 29 11:01:40 dist-test-slave-2lzq krb5kdc[7919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679700, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.252@KRBTEST.COM for kudu/127.26.120.252@KRBTEST.COM
I20260929 11:01:40.276965 8003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:40.269659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55836 (local address 127.26.120.254:43835)
0929 11:01:40.269699 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:40.269700 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:40.269748 (+ 48us) server_negotiation.cc:407] Connection header received
0929 11:01:40.269819 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:40.269820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:40.269828 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:40.269871 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:01:40.270483 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.271145 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.271823 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.272035 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.272117 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:40.274331 (+ 2214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:01:40.274332 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:01:40.274332 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:01:40.274338 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:01:40.275889 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:40.276207 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:40.276208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:40.276209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:40.276262 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:40.276468 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:40.276468 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:40.276469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:40.276578 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:01:40.276764 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:40.276801 (+ 37us) server_negotiation.cc:299] Negotiation successful
0929 11:01:40.276875 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 11:01:40.277385 8153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:40.271374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:45869 (local address 127.0.0.1:44018)
0929 11:01:40.271403 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:40.271412 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:40.271446 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:40.271555 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:40.271556 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:40.271682 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:01:40.271704 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:40.271706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.272361 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.272362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:40.272814 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:40.272815 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.273193 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.273194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:40.273243 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:40.273588 (+ 345us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:01:40.273591 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:01:40.274791 (+ 1200us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:01:40.276388 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:40.276389 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:40.276390 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:40.276551 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:40.276741 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:40.276742 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:40.276743 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:40.276794 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:40.277052 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:01:40.277053 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:01:40.277088 (+ 35us) client_negotiation.cc:769] Sending connection context
0929 11:01:40.277176 (+ 88us) client_negotiation.cc:240] Negotiation successful
0929 11:01:40.277291 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14}
I20260929 11:01:40.277606 8155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:40.271405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44018 (local address 127.26.120.252:45869)
0929 11:01:40.271464 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:40.271466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:40.271473 (+ 7us) server_negotiation.cc:407] Connection header received
0929 11:01:40.271491 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:40.271492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:40.271500 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:40.271555 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:01:40.271783 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.272265 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.272909 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.273113 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.273192 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:40.274884 (+ 1692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:01:40.274885 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:01:40.274885 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:01:40.274891 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:01:40.276295 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:40.276630 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:40.276631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:40.276631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:40.276681 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:40.276867 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:40.276867 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:40.276868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:40.276990 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:01:40.277049 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:40.277197 (+ 148us) server_negotiation.cc:299] Negotiation successful
0929 11:01:40.277358 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":77,"server-negotiator.queue_time_us":45}
I20260929 11:01:40.279131 8151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1c80174493e4a04b785197fcde474e3: Bootstrap starting.
I20260929 11:01:40.280046 8151 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1c80174493e4a04b785197fcde474e3: Neither blocks nor log segments found. Creating new log.
I20260929 11:01:40.280386 8151 log.cc:824] T 00000000000000000000000000000000 P c1c80174493e4a04b785197fcde474e3: Log is configured to *not* fsync() on all Append() calls
I20260929 11:01:40.281111 8151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1c80174493e4a04b785197fcde474e3: No bootstrap required, opened a new log
Sep 29 11:01:40 dist-test-slave-2lzq krb5kdc[7919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.253@KRBTEST.COM
I20260929 11:01:40.283322 8151 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1c80174493e4a04b785197fcde474e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4538f5ba711a46b6854bd148ec45cb64" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43835 } } peers { permanent_uuid: "39bdbda009ed43d0956c70d48a5bff8b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 36357 } } peers { permanent_uuid: "c1c80174493e4a04b785197fcde474e3" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 45869 } }
I20260929 11:01:40.283485 8151 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1c80174493e4a04b785197fcde474e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:01:40.283526 8151 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1c80174493e4a04b785197fcde474e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1c80174493e4a04b785197fcde474e3, State: Initialized, Role: FOLLOWER
I20260929 11:01:40.283650 8151 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1c80174493e4a04b785197fcde474e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4538f5ba711a46b6854bd148ec45cb64" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43835 } } peers { permanent_uuid: "39bdbda009ed43d0956c70d48a5bff8b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 36357 } } peers { permanent_uuid: "c1c80174493e4a04b785197fcde474e3" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 45869 } }
I20260929 11:01:40.283919 8157 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1c80174493e4a04b785197fcde474e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4538f5ba711a46b6854bd148ec45cb64" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43835 } } peers { permanent_uuid: "39bdbda009ed43d0956c70d48a5bff8b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 36357 } } peers { permanent_uuid: "c1c80174493e4a04b785197fcde474e3" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 45869 } } }
I20260929 11:01:40.283972 8151 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1c80174493e4a04b785197fcde474e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:01:40.284035 8157 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1c80174493e4a04b785197fcde474e3 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 11:01:40.285354 8080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:40.277522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46780 (local address 127.26.120.253:36357)
0929 11:01:40.277561 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:40.277563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:40.277636 (+ 73us) server_negotiation.cc:407] Connection header received
0929 11:01:40.277735 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:40.277736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:40.277746 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:40.277804 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:01:40.278444 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.279103 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.279780 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.280072 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.280145 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:40.282393 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:01:40.282394 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:01:40.282394 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:01:40.282401 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:01:40.283982 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:40.284360 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:40.284361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:40.284362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:40.284414 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:40.284637 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:40.284638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:40.284638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:40.284758 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:01:40.284915 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:40.285161 (+ 246us) server_negotiation.cc:299] Negotiation successful
0929 11:01:40.285247 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260929 11:01:40.285931 8168 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c1c80174493e4a04b785197fcde474e3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 11:01:40.285988 8168 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 11:01:40.286844 27107 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 11:01:40.287510 8001 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.120.252" port: 45869 } attempt: 2
I20260929 11:01:40.286295 8083 master_runner.cc:186] Error getting master registration for 127.26.120.254:43835: OK, instance_id { permanent_uuid: "4538f5ba711a46b6854bd148ec45cb64" instance_seqno: 1790679699944391 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 11:01:40.288579 8083 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Sep 29 11:01:40 dist-test-slave-2lzq krb5kdc[7919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679699, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for kudu/127.26.120.252@KRBTEST.COM
I20260929 11:01:40.295580 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:40.287760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:45869 (local address 127.0.0.1:44020)
0929 11:01:40.287841 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:40.287851 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:40.287887 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:40.288131 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:40.288132 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:40.288283 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:01:40.288325 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:40.288327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.289579 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.289580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:40.290151 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:40.290154 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.290684 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.290685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:40.290754 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:40.291194 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:01:40.291198 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:01:40.292619 (+ 1421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:01:40.294431 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:40.294431 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:40.294432 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:40.294660 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:40.294886 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:40.294887 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:40.294887 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:40.294949 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:40.295284 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:01:40.295285 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:01:40.295329 (+ 44us) client_negotiation.cc:769] Sending connection context
0929 11:01:40.295378 (+ 49us) client_negotiation.cc:240] Negotiation successful
0929 11:01:40.295461 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260929 11:01:40.295957 8155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:40.287962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44020 (local address 127.26.120.252:45869)
0929 11:01:40.288022 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:40.288024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:40.288030 (+ 6us) server_negotiation.cc:407] Connection header received
0929 11:01:40.288047 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:40.288048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:40.288057 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:40.288510 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:01:40.288542 (+ 32us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.289478 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.290246 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.290591 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.290906 (+ 315us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:40.292706 (+ 1800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:01:40.292707 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:01:40.292708 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:01:40.292714 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:01:40.294353 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:40.294748 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:40.294749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:40.294749 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:40.294804 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:40.295037 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:40.295038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:40.295039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:40.295200 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:01:40.295739 (+ 539us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:40.295781 (+ 42us) server_negotiation.cc:299] Negotiation successful
0929 11:01:40.295857 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 11:01:40.297593 8001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4538f5ba711a46b6854bd148ec45cb64: Bootstrap starting.
I20260929 11:01:40.297940 8080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:40.288436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46786 (local address 127.26.120.253:36357)
0929 11:01:40.288472 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:40.288474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:40.288760 (+ 286us) server_negotiation.cc:407] Connection header received
0929 11:01:40.288851 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:40.288852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:40.288860 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:40.288888 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:01:40.289587 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.290311 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.291707 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.291994 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.292155 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:40.294435 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:01:40.294436 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:01:40.294437 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:01:40.294444 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:01:40.296603 (+ 2159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:40.296994 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:40.296995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:40.296995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:40.297065 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:40.297310 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:40.297311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:40.297312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:40.297440 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:01:40.297517 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:40.297761 (+ 244us) server_negotiation.cc:299] Negotiation successful
0929 11:01:40.297848 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 11:01:40.298122 8003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:40.288881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55852 (local address 127.26.120.254:43835)
0929 11:01:40.288921 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:40.288923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:40.290207 (+ 1284us) server_negotiation.cc:407] Connection header received
0929 11:01:40.290300 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:40.290301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:40.290309 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:40.290334 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:01:40.291077 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.291749 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.292798 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.293187 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.293255 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:40.295321 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:01:40.295322 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:01:40.295323 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:01:40.295329 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:01:40.296852 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:40.297235 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:40.297236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:40.297236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:40.297307 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:40.297531 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:40.297531 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:40.297532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:40.297649 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:01:40.297717 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:40.297920 (+ 203us) server_negotiation.cc:299] Negotiation successful
0929 11:01:40.298002 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 11:01:40.298412 8001 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4538f5ba711a46b6854bd148ec45cb64: Neither blocks nor log segments found. Creating new log.
I20260929 11:01:40.298741 8001 log.cc:824] T 00000000000000000000000000000000 P 4538f5ba711a46b6854bd148ec45cb64: Log is configured to *not* fsync() on all Append() calls
I20260929 11:01:40.299466 8001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4538f5ba711a46b6854bd148ec45cb64: No bootstrap required, opened a new log
I20260929 11:01:40.300765 8001 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4538f5ba711a46b6854bd148ec45cb64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4538f5ba711a46b6854bd148ec45cb64" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43835 } } peers { permanent_uuid: "39bdbda009ed43d0956c70d48a5bff8b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 36357 } } peers { permanent_uuid: "c1c80174493e4a04b785197fcde474e3" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 45869 } }
I20260929 11:01:40.300871 8001 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4538f5ba711a46b6854bd148ec45cb64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:01:40.300894 8001 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4538f5ba711a46b6854bd148ec45cb64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4538f5ba711a46b6854bd148ec45cb64, State: Initialized, Role: FOLLOWER
I20260929 11:01:40.301014 8001 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4538f5ba711a46b6854bd148ec45cb64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4538f5ba711a46b6854bd148ec45cb64" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43835 } } peers { permanent_uuid: "39bdbda009ed43d0956c70d48a5bff8b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 36357 } } peers { permanent_uuid: "c1c80174493e4a04b785197fcde474e3" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 45869 } }
I20260929 11:01:40.301256 8179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4538f5ba711a46b6854bd148ec45cb64 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4538f5ba711a46b6854bd148ec45cb64" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43835 } } peers { permanent_uuid: "39bdbda009ed43d0956c70d48a5bff8b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 36357 } } peers { permanent_uuid: "c1c80174493e4a04b785197fcde474e3" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 45869 } } }
I20260929 11:01:40.301307 8001 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4538f5ba711a46b6854bd148ec45cb64 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:01:40.301327 8179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4538f5ba711a46b6854bd148ec45cb64 [sys.catalog]: This master's current role is: FOLLOWER
W20260929 11:01:40.303025 8190 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4538f5ba711a46b6854bd148ec45cb64: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 11:01:40.303081 8190 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 11:01:40.304015 7932 master_runner.cc:186] Error getting master registration for 127.26.120.253:36357: OK, instance_id { permanent_uuid: "39bdbda009ed43d0956c70d48a5bff8b" instance_seqno: 1790679700097203 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 11:01:40.306468 7932 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 11:01:40.308915 8176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:40.288615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44022 (local address 127.26.120.252:45869)
0929 11:01:40.290998 (+ 2383us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:40.290999 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:40.291006 (+ 7us) server_negotiation.cc:407] Connection header received
0929 11:01:40.291023 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:40.291024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:40.291034 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:40.291143 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:01:40.293938 (+ 2795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.303511 (+ 9573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.304501 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.304828 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.304890 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:40.306396 (+ 1506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:01:40.306397 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:01:40.306398 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:01:40.306404 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:01:40.307726 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:40.308103 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:40.308104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:40.308104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:40.308162 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:40.308361 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:40.308362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:40.308362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:40.308472 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:01:40.308571 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:40.308709 (+ 138us) server_negotiation.cc:299] Negotiation successful
0929 11:01:40.308787 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2353,"thread_start_us":89,"threads_started":1}
I20260929 11:01:40.333585 8076 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.120.252" port: 45869 } attempt: 3
Sep 29 11:01:40 dist-test-slave-2lzq krb5kdc[7919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679700, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.253@KRBTEST.COM for kudu/127.26.120.252@KRBTEST.COM
I20260929 11:01:40.341461 8176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:40.333980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44030 (local address 127.26.120.252:45869)
0929 11:01:40.334028 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:40.334030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:40.334039 (+ 9us) server_negotiation.cc:407] Connection header received
0929 11:01:40.334165 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:40.334166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:40.334176 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:40.334246 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:01:40.334597 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.335221 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.335939 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.336200 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.336265 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:40.338641 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:01:40.338642 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:01:40.338642 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:01:40.338649 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:01:40.340290 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:40.340645 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:40.340647 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:40.340647 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:40.340722 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:40.340925 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:40.340926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:40.340926 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:40.341054 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:01:40.341127 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:40.341294 (+ 167us) server_negotiation.cc:299] Negotiation successful
0929 11:01:40.341373 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260929 11:01:40.341461 8078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:40.333927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:45869 (local address 127.0.0.1:44030)
0929 11:01:40.333983 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:40.333995 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:40.334040 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:40.334252 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:40.334253 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:40.334454 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:01:40.334507 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:40.334509 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.335311 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.335313 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:40.335851 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:40.335853 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:40.336260 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:40.336261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:40.336345 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:40.336893 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:01:40.336899 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:01:40.338535 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:01:40.340388 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:40.340389 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:40.340389 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:40.340563 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:40.340791 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:40.340792 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:40.340793 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:40.340852 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:40.341129 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:01:40.341130 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:01:40.341176 (+ 46us) client_negotiation.cc:769] Sending connection context
0929 11:01:40.341261 (+ 85us) client_negotiation.cc:240] Negotiation successful
0929 11:01:40.341363 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260929 11:01:40.343098 8076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39bdbda009ed43d0956c70d48a5bff8b: Bootstrap starting.
I20260929 11:01:40.343798 8076 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39bdbda009ed43d0956c70d48a5bff8b: Neither blocks nor log segments found. Creating new log.
I20260929 11:01:40.344106 8076 log.cc:824] T 00000000000000000000000000000000 P 39bdbda009ed43d0956c70d48a5bff8b: Log is configured to *not* fsync() on all Append() calls
I20260929 11:01:40.344977 8076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39bdbda009ed43d0956c70d48a5bff8b: No bootstrap required, opened a new log
I20260929 11:01:40.346607 8076 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39bdbda009ed43d0956c70d48a5bff8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4538f5ba711a46b6854bd148ec45cb64" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43835 } } peers { permanent_uuid: "39bdbda009ed43d0956c70d48a5bff8b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 36357 } } peers { permanent_uuid: "c1c80174493e4a04b785197fcde474e3" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 45869 } }
I20260929 11:01:40.346745 8076 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39bdbda009ed43d0956c70d48a5bff8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:01:40.346797 8076 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39bdbda009ed43d0956c70d48a5bff8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39bdbda009ed43d0956c70d48a5bff8b, State: Initialized, Role: FOLLOWER
I20260929 11:01:40.346935 8076 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39bdbda009ed43d0956c70d48a5bff8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4538f5ba711a46b6854bd148ec45cb64" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43835 } } peers { permanent_uuid: "39bdbda009ed43d0956c70d48a5bff8b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 36357 } } peers { permanent_uuid: "c1c80174493e4a04b785197fcde474e3" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 45869 } }
I20260929 11:01:40.347216 8193 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39bdbda009ed43d0956c70d48a5bff8b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4538f5ba711a46b6854bd148ec45cb64" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43835 } } peers { permanent_uuid: "39bdbda009ed43d0956c70d48a5bff8b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 36357 } } peers { permanent_uuid: "c1c80174493e4a04b785197fcde474e3" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 45869 } } }
I20260929 11:01:40.347361 8193 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39bdbda009ed43d0956c70d48a5bff8b [sys.catalog]: This master's current role is: FOLLOWER
I20260929 11:01:40.347278 8076 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39bdbda009ed43d0956c70d48a5bff8b [sys.catalog]: configured and running, proceeding with master startup.
W20260929 11:01:40.349327 8204 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 39bdbda009ed43d0956c70d48a5bff8b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 11:01:40.349436 8204 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 11:01:40.351398 8008 master_runner.cc:263] No leader master found from master 39bdbda009ed43d0956c70d48a5bff8b
I20260929 11:01:40.351454 8008 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260929 11:01:40.442391 7953 tablet.cc:2369] T 00000000000000000000000000000000 P 4538f5ba711a46b6854bd148ec45cb64: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 11:01:40.497828 8104 tablet.cc:2369] T 00000000000000000000000000000000 P c1c80174493e4a04b785197fcde474e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 11:01:40.595032 8029 tablet.cc:2369] T 00000000000000000000000000000000 P 39bdbda009ed43d0956c70d48a5bff8b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 11:01:40.653879 8157 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1c80174493e4a04b785197fcde474e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 11:01:40.654055 8157 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1c80174493e4a04b785197fcde474e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4538f5ba711a46b6854bd148ec45cb64" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43835 } } peers { permanent_uuid: "39bdbda009ed43d0956c70d48a5bff8b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 36357 } } peers { permanent_uuid: "c1c80174493e4a04b785197fcde474e3" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 45869 } }
I20260929 11:01:40.654376 8157 leader_election.cc:288] T 00000000000000000000000000000000 P c1c80174493e4a04b785197fcde474e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4538f5ba711a46b6854bd148ec45cb64 (127.26.120.254:43835), 39bdbda009ed43d0956c70d48a5bff8b (127.26.120.253:36357)
I20260929 11:01:40.654732 7974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c1c80174493e4a04b785197fcde474e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4538f5ba711a46b6854bd148ec45cb64" is_pre_election: true
I20260929 11:01:40.654932 7974 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4538f5ba711a46b6854bd148ec45cb64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1c80174493e4a04b785197fcde474e3 in term 0.
I20260929 11:01:40.654956 8050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c1c80174493e4a04b785197fcde474e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39bdbda009ed43d0956c70d48a5bff8b" is_pre_election: true
I20260929 11:01:40.655131 8050 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 39bdbda009ed43d0956c70d48a5bff8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1c80174493e4a04b785197fcde474e3 in term 0.
I20260929 11:01:40.655143 8099 leader_election.cc:302] T 00000000000000000000000000000000 P c1c80174493e4a04b785197fcde474e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4538f5ba711a46b6854bd148ec45cb64, c1c80174493e4a04b785197fcde474e3; no voters:
I20260929 11:01:40.655392 8157 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1c80174493e4a04b785197fcde474e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 11:01:40.655478 8157 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1c80174493e4a04b785197fcde474e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 11:01:40.655501 8157 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1c80174493e4a04b785197fcde474e3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:01:40.656256 8157 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1c80174493e4a04b785197fcde474e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4538f5ba711a46b6854bd148ec45cb64" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43835 } } peers { permanent_uuid: "39bdbda009ed43d0956c70d48a5bff8b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 36357 } } peers { permanent_uuid: "c1c80174493e4a04b785197fcde474e3" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 45869 } }
I20260929 11:01:40.656401 8157 leader_election.cc:288] T 00000000000000000000000000000000 P c1c80174493e4a04b785197fcde474e3 [CANDIDATE]: Term 1 election: Requested vote from peers 4538f5ba711a46b6854bd148ec45cb64 (127.26.120.254:43835), 39bdbda009ed43d0956c70d48a5bff8b (127.26.120.253:36357)
I20260929 11:01:40.656744 7974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c1c80174493e4a04b785197fcde474e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4538f5ba711a46b6854bd148ec45cb64"
I20260929 11:01:40.656848 7974 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4538f5ba711a46b6854bd148ec45cb64 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:01:40.657581 7974 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4538f5ba711a46b6854bd148ec45cb64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1c80174493e4a04b785197fcde474e3 in term 1.
I20260929 11:01:40.657722 8050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c1c80174493e4a04b785197fcde474e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39bdbda009ed43d0956c70d48a5bff8b"
I20260929 11:01:40.657809 8099 leader_election.cc:302] T 00000000000000000000000000000000 P c1c80174493e4a04b785197fcde474e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4538f5ba711a46b6854bd148ec45cb64, c1c80174493e4a04b785197fcde474e3; no voters:
I20260929 11:01:40.657820 8050 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39bdbda009ed43d0956c70d48a5bff8b [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:01:40.657886 8157 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1c80174493e4a04b785197fcde474e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:01:40.658001 8157 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1c80174493e4a04b785197fcde474e3 [term 1 LEADER]: Becoming Leader. State: Replica: c1c80174493e4a04b785197fcde474e3, State: Running, Role: LEADER
I20260929 11:01:40.658130 8157 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1c80174493e4a04b785197fcde474e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4538f5ba711a46b6854bd148ec45cb64" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43835 } } peers { permanent_uuid: "39bdbda009ed43d0956c70d48a5bff8b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 36357 } } peers { permanent_uuid: "c1c80174493e4a04b785197fcde474e3" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 45869 } }
I20260929 11:01:40.658582 8206 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1c80174493e4a04b785197fcde474e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1c80174493e4a04b785197fcde474e3. Latest consensus state: current_term: 1 leader_uuid: "c1c80174493e4a04b785197fcde474e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4538f5ba711a46b6854bd148ec45cb64" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43835 } } peers { permanent_uuid: "39bdbda009ed43d0956c70d48a5bff8b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 36357 } } peers { permanent_uuid: "c1c80174493e4a04b785197fcde474e3" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 45869 } } }
I20260929 11:01:40.658679 8206 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1c80174493e4a04b785197fcde474e3 [sys.catalog]: This master's current role is: LEADER
I20260929 11:01:40.658859 8050 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 39bdbda009ed43d0956c70d48a5bff8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1c80174493e4a04b785197fcde474e3 in term 1.
I20260929 11:01:40.658876 8208 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:01:40.659067 8208 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:01:40.660440 8050 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 39bdbda009ed43d0956c70d48a5bff8b [term 1 FOLLOWER]: Refusing update from remote peer c1c80174493e4a04b785197fcde474e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 11:01:40.660452 7974 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4538f5ba711a46b6854bd148ec45cb64 [term 1 FOLLOWER]: Refusing update from remote peer c1c80174493e4a04b785197fcde474e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 11:01:40.660802 8157 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c1c80174493e4a04b785197fcde474e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4538f5ba711a46b6854bd148ec45cb64" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 11:01:40.660908 8206 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c1c80174493e4a04b785197fcde474e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39bdbda009ed43d0956c70d48a5bff8b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 36357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 11:01:40.661922 8193 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39bdbda009ed43d0956c70d48a5bff8b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1c80174493e4a04b785197fcde474e3. Latest consensus state: current_term: 1 leader_uuid: "c1c80174493e4a04b785197fcde474e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4538f5ba711a46b6854bd148ec45cb64" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43835 } } peers { permanent_uuid: "39bdbda009ed43d0956c70d48a5bff8b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 36357 } } peers { permanent_uuid: "c1c80174493e4a04b785197fcde474e3" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 45869 } } }
I20260929 11:01:40.662026 8193 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39bdbda009ed43d0956c70d48a5bff8b [sys.catalog]: This master's current role is: FOLLOWER
I20260929 11:01:40.662122 8179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4538f5ba711a46b6854bd148ec45cb64 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1c80174493e4a04b785197fcde474e3. Latest consensus state: current_term: 1 leader_uuid: "c1c80174493e4a04b785197fcde474e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4538f5ba711a46b6854bd148ec45cb64" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43835 } } peers { permanent_uuid: "39bdbda009ed43d0956c70d48a5bff8b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 36357 } } peers { permanent_uuid: "c1c80174493e4a04b785197fcde474e3" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 45869 } } }
I20260929 11:01:40.662199 8179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4538f5ba711a46b6854bd148ec45cb64 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 11:01:40.663031 8157 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1c80174493e4a04b785197fcde474e3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c1c80174493e4a04b785197fcde474e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4538f5ba711a46b6854bd148ec45cb64" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43835 } } peers { permanent_uuid: "39bdbda009ed43d0956c70d48a5bff8b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 36357 } } peers { permanent_uuid: "c1c80174493e4a04b785197fcde474e3" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 45869 } } }
I20260929 11:01:40.663132 8157 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1c80174493e4a04b785197fcde474e3 [sys.catalog]: This master's current role is: LEADER
I20260929 11:01:40.663169 8179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4538f5ba711a46b6854bd148ec45cb64 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c1c80174493e4a04b785197fcde474e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4538f5ba711a46b6854bd148ec45cb64" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43835 } } peers { permanent_uuid: "39bdbda009ed43d0956c70d48a5bff8b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 36357 } } peers { permanent_uuid: "c1c80174493e4a04b785197fcde474e3" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 45869 } } }
I20260929 11:01:40.663249 8179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4538f5ba711a46b6854bd148ec45cb64 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 11:01:40.663278 8157 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1c80174493e4a04b785197fcde474e3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c1c80174493e4a04b785197fcde474e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4538f5ba711a46b6854bd148ec45cb64" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43835 } } peers { permanent_uuid: "39bdbda009ed43d0956c70d48a5bff8b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 36357 } } peers { permanent_uuid: "c1c80174493e4a04b785197fcde474e3" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 45869 } } }
I20260929 11:01:40.663357 8157 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1c80174493e4a04b785197fcde474e3 [sys.catalog]: This master's current role is: LEADER
I20260929 11:01:40.663316 8208 catalog_manager.cc:1347] Generated new cluster ID: 4a8e6ad7db09471b969a48389f0cf45f
I20260929 11:01:40.663519 8208 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 11:01:40.663805 8193 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39bdbda009ed43d0956c70d48a5bff8b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c1c80174493e4a04b785197fcde474e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4538f5ba711a46b6854bd148ec45cb64" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43835 } } peers { permanent_uuid: "39bdbda009ed43d0956c70d48a5bff8b" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 36357 } } peers { permanent_uuid: "c1c80174493e4a04b785197fcde474e3" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 45869 } } }
I20260929 11:01:40.663879 8193 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39bdbda009ed43d0956c70d48a5bff8b [sys.catalog]: This master's current role is: FOLLOWER
I20260929 11:01:40.685047 8208 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 11:01:40.685731 8208 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:01:40.691743 8208 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1c80174493e4a04b785197fcde474e3: Generated new TSK 0
I20260929 11:01:40.691951 8208 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 11:01:41.303519 8190 catalog_manager.cc:1259] Loaded cluster ID: 4a8e6ad7db09471b969a48389f0cf45f
I20260929 11:01:41.303597 8190 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4538f5ba711a46b6854bd148ec45cb64: loading cluster ID for follower catalog manager: success
I20260929 11:01:41.304396 8190 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4538f5ba711a46b6854bd148ec45cb64: acquiring CA information for follower catalog manager: success
I20260929 11:01:41.304639 8190 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4538f5ba711a46b6854bd148ec45cb64: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 11:01:41.349896 8204 catalog_manager.cc:1259] Loaded cluster ID: 4a8e6ad7db09471b969a48389f0cf45f
I20260929 11:01:41.349973 8204 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 39bdbda009ed43d0956c70d48a5bff8b: loading cluster ID for follower catalog manager: success
I20260929 11:01:41.350805 8204 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 39bdbda009ed43d0956c70d48a5bff8b: acquiring CA information for follower catalog manager: success
I20260929 11:01:41.351065 8204 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 39bdbda009ed43d0956c70d48a5bff8b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 11:01:41.743094 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 7932
I20260929 11:01:41.748354 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 8008
I20260929 11:01:41.753091 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 8083
2026-09-29T11:01:41Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3061 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 11:01:41 dist-test-slave-2lzq krb5kdc[8224](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 11:01:41 dist-test-slave-2lzq krb5kdc[8224](info): set up 2 sockets
Sep 29 11:01:41 dist-test-slave-2lzq krb5kdc[8224](info): commencing operation
krb5kdc: starting...
W20260929 11:01:42.802606 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 11:01:42 dist-test-slave-2lzq krb5kdc[8224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T11:01:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T11:01:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.120.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
I20260929 11:01:42.920921 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:43927
--webserver_interface=127.26.120.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:46065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:43927
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:01:43.002578 8237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:01:43.002815 8237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:01:43.002852 8237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:01:43.004004 8237 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:01:43.004068 8237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:01:43.004096 8237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:01:43.004124 8237 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:01:43.004148 8237 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:01:43.005611 8237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:46065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:43927
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:43927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:01:43.005880 8237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:01:43.006152 8237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:01:43.010957 8245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:01:43.010957 8242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:01:43.010994 8243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:01:43.011204 8237 server_base.cc:1034] running on GCE node
I20260929 11:01:43.011495 8237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:01:43.011752 8237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:01:43.012930 8237 hybrid_clock.cc:648] HybridClock initialized: now 1790679703012924 us; error 51 us; skew 500 ppm
Sep 29 11:01:43 dist-test-slave-2lzq krb5kdc[8224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679703, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:01:43.015491 8237 init.cc:377] Logged in from keytab as kudu/127.26.120.254@KRBTEST.COM (short username kudu)
I20260929 11:01:43.015954 8237 webserver.cc:460] Webserver started at http://127.26.120.254:33729/ using document root <none> and password file <none>
I20260929 11:01:43.016140 8237 fs_manager.cc:359] Metadata directory not provided
I20260929 11:01:43.016180 8237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:01:43.016280 8237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:01:43.017380 8237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "09a0c979b7ba41dea31b5aa642b8563e"
format_stamp: "Formatted at 2026-09-29 11:01:43 on dist-test-slave-2lzq"
server_key: "ac4f9fb053601f3d193832ac742a224a"
server_key_iv: "12346ce626461e3de6e3d74ae4cbf47f"
server_key_version: "encryptionkey@0"
I20260929 11:01:43.017688 8237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "09a0c979b7ba41dea31b5aa642b8563e"
format_stamp: "Formatted at 2026-09-29 11:01:43 on dist-test-slave-2lzq"
server_key: "ac4f9fb053601f3d193832ac742a224a"
server_key_iv: "12346ce626461e3de6e3d74ae4cbf47f"
server_key_version: "encryptionkey@0"
I20260929 11:01:43.019065 8237 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 11:01:43.019753 8252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:43.019920 8237 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 11:01:43.020009 8237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "09a0c979b7ba41dea31b5aa642b8563e"
format_stamp: "Formatted at 2026-09-29 11:01:43 on dist-test-slave-2lzq"
server_key: "ac4f9fb053601f3d193832ac742a224a"
server_key_iv: "12346ce626461e3de6e3d74ae4cbf47f"
server_key_version: "encryptionkey@0"
I20260929 11:01:43.020087 8237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:01:43.029223 8237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:01:43.031085 8237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:01:43.031237 8237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:01:43.035092 8237 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:43927
I20260929 11:01:43.035145 8304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:43927 every 8 connection(s)
I20260929 11:01:43.035463 8237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:01:43.035784 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 8237
I20260929 11:01:43.035926 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 11:01:43.036125 27107 external_mini_cluster.cc:1428] Setting key 8665b59a794a3517331218865e000860
I20260929 11:01:43.036564 8305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:01:43.039719 8305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09a0c979b7ba41dea31b5aa642b8563e: Bootstrap starting.
I20260929 11:01:43.040349 8305 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09a0c979b7ba41dea31b5aa642b8563e: Neither blocks nor log segments found. Creating new log.
I20260929 11:01:43.040652 8305 log.cc:824] T 00000000000000000000000000000000 P 09a0c979b7ba41dea31b5aa642b8563e: Log is configured to *not* fsync() on all Append() calls
I20260929 11:01:43.041383 8305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09a0c979b7ba41dea31b5aa642b8563e: No bootstrap required, opened a new log
Sep 29 11:01:43 dist-test-slave-2lzq krb5kdc[8224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:01:43.043417 8305 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09a0c979b7ba41dea31b5aa642b8563e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09a0c979b7ba41dea31b5aa642b8563e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43927 } }
I20260929 11:01:43.043604 8305 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09a0c979b7ba41dea31b5aa642b8563e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:01:43.043689 8305 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09a0c979b7ba41dea31b5aa642b8563e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09a0c979b7ba41dea31b5aa642b8563e, State: Initialized, Role: FOLLOWER
I20260929 11:01:43.043874 8305 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09a0c979b7ba41dea31b5aa642b8563e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09a0c979b7ba41dea31b5aa642b8563e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43927 } }
I20260929 11:01:43.043972 8305 raft_consensus.cc:388] T 00000000000000000000000000000000 P 09a0c979b7ba41dea31b5aa642b8563e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 11:01:43.044016 8305 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09a0c979b7ba41dea31b5aa642b8563e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 11:01:43.044066 8305 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09a0c979b7ba41dea31b5aa642b8563e [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:01:43.044582 8305 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09a0c979b7ba41dea31b5aa642b8563e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09a0c979b7ba41dea31b5aa642b8563e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43927 } }
I20260929 11:01:43.044698 8305 leader_election.cc:302] T 00000000000000000000000000000000 P 09a0c979b7ba41dea31b5aa642b8563e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 09a0c979b7ba41dea31b5aa642b8563e; no voters:
I20260929 11:01:43.044888 8305 leader_election.cc:288] T 00000000000000000000000000000000 P 09a0c979b7ba41dea31b5aa642b8563e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 11:01:43.044931 8310 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09a0c979b7ba41dea31b5aa642b8563e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:01:43.045121 8310 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09a0c979b7ba41dea31b5aa642b8563e [term 1 LEADER]: Becoming Leader. State: Replica: 09a0c979b7ba41dea31b5aa642b8563e, State: Running, Role: LEADER
I20260929 11:01:43.045169 8305 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09a0c979b7ba41dea31b5aa642b8563e [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:01:43.045306 8310 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09a0c979b7ba41dea31b5aa642b8563e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09a0c979b7ba41dea31b5aa642b8563e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43927 } }
I20260929 11:01:43.045728 8310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09a0c979b7ba41dea31b5aa642b8563e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09a0c979b7ba41dea31b5aa642b8563e. Latest consensus state: current_term: 1 leader_uuid: "09a0c979b7ba41dea31b5aa642b8563e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09a0c979b7ba41dea31b5aa642b8563e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43927 } } }
I20260929 11:01:43.045953 8310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09a0c979b7ba41dea31b5aa642b8563e [sys.catalog]: This master's current role is: LEADER
I20260929 11:01:43.045882 8312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09a0c979b7ba41dea31b5aa642b8563e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "09a0c979b7ba41dea31b5aa642b8563e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09a0c979b7ba41dea31b5aa642b8563e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43927 } } }
I20260929 11:01:43.046156 8312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09a0c979b7ba41dea31b5aa642b8563e [sys.catalog]: This master's current role is: LEADER
I20260929 11:01:43.046598 8319 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:01:43.047070 8308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:43.036617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55500 (local address 127.26.120.254:43927)
0929 11:01:43.036765 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:43.036774 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:43.036809 (+ 35us) server_negotiation.cc:407] Connection header received
0929 11:01:43.037233 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:43.037239 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:43.037296 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:43.037405 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:01:43.038182 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:43.039049 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:43.040219 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:43.040707 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:43.040828 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:43.043202 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:01:43.043206 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:01:43.043207 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:01:43.043219 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:01:43.045217 (+ 1998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:43.045605 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:43.045608 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:43.045608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:43.045712 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:43.045933 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:43.045934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:43.045934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:43.046274 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:01:43.046605 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:43.046691 (+ 86us) server_negotiation.cc:299] Negotiation successful
0929 11:01:43.046851 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":74,"threads_started":1}
I20260929 11:01:43.047506 8319 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:01:43.047991 27107 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 11:01:43.049576 8319 catalog_manager.cc:1347] Generated new cluster ID: 5b247734afa046538be4043ce65a8ced
I20260929 11:01:43.049628 8319 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 11:01:43.068480 8319 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 11:01:43.069036 8319 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:01:43.072072 8319 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09a0c979b7ba41dea31b5aa642b8563e: Generated new TSK 0
I20260929 11:01:43.072283 8319 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 11:01:43.102016 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 8237
2026-09-29T11:01:43Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1347 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 11:01:43 dist-test-slave-2lzq krb5kdc[8332](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 11:01:43 dist-test-slave-2lzq krb5kdc[8332](info): set up 2 sockets
Sep 29 11:01:43 dist-test-slave-2lzq krb5kdc[8332](info): commencing operation
krb5kdc: starting...
W20260929 11:01:44.126456 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 11:01:44 dist-test-slave-2lzq krb5kdc[8332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T11:01:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T11:01:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.120.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
I20260929 11:01:44.245374 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:43375
--webserver_interface=127.26.120.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:33997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:43375
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:01:44.325852 8345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:01:44.326116 8345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:01:44.326157 8345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:01:44.327296 8345 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:01:44.327366 8345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:01:44.327401 8345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:01:44.327433 8345 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:01:44.327466 8345 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:01:44.328918 8345 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:33997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:43375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:43375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:01:44.329197 8345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:01:44.329434 8345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:01:44.334513 8353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:01:44.334558 8345 server_base.cc:1034] running on GCE node
W20260929 11:01:44.334529 8351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:01:44.334533 8350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:01:44.334978 8345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:01:44.335232 8345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:01:44.336372 8345 hybrid_clock.cc:648] HybridClock initialized: now 1790679704336361 us; error 27 us; skew 500 ppm
Sep 29 11:01:44 dist-test-slave-2lzq krb5kdc[8332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679704, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:01:44.338833 8345 init.cc:377] Logged in from keytab as kudu/127.26.120.254@KRBTEST.COM (short username kudu)
I20260929 11:01:44.339326 8345 webserver.cc:460] Webserver started at http://127.26.120.254:37609/ using document root <none> and password file <none>
I20260929 11:01:44.339555 8345 fs_manager.cc:359] Metadata directory not provided
I20260929 11:01:44.339598 8345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:01:44.339775 8345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:01:44.340813 8345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "099a625a94ff4147881504b477a5305e"
format_stamp: "Formatted at 2026-09-29 11:01:44 on dist-test-slave-2lzq"
server_key: "f7b320c7754e8e63580725832f3b3fb3"
server_key_iv: "17f2f79eeaebb4ed7329cc26bc7bbd10"
server_key_version: "encryptionkey@0"
I20260929 11:01:44.341164 8345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "099a625a94ff4147881504b477a5305e"
format_stamp: "Formatted at 2026-09-29 11:01:44 on dist-test-slave-2lzq"
server_key: "f7b320c7754e8e63580725832f3b3fb3"
server_key_iv: "17f2f79eeaebb4ed7329cc26bc7bbd10"
server_key_version: "encryptionkey@0"
I20260929 11:01:44.342566 8345 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260929 11:01:44.343358 8360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:44.343544 8345 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 11:01:44.343652 8345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "099a625a94ff4147881504b477a5305e"
format_stamp: "Formatted at 2026-09-29 11:01:44 on dist-test-slave-2lzq"
server_key: "f7b320c7754e8e63580725832f3b3fb3"
server_key_iv: "17f2f79eeaebb4ed7329cc26bc7bbd10"
server_key_version: "encryptionkey@0"
I20260929 11:01:44.343729 8345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:01:44.359498 8345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:01:44.361366 8345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:01:44.361533 8345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:01:44.365520 8345 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:43375
I20260929 11:01:44.365623 8412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:43375 every 8 connection(s)
I20260929 11:01:44.365952 8345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:01:44.366937 8413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:01:44.369181 8413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e: Bootstrap starting.
I20260929 11:01:44.369724 8413 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e: Neither blocks nor log segments found. Creating new log.
I20260929 11:01:44.369951 8413 log.cc:824] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e: Log is configured to *not* fsync() on all Append() calls
I20260929 11:01:44.370299 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 8345
I20260929 11:01:44.370414 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 11:01:44.370609 27107 external_mini_cluster.cc:1428] Setting key dd990aed5f64a449722d0fa905111599
I20260929 11:01:44.370815 8413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e: No bootstrap required, opened a new log
I20260929 11:01:44.372630 8413 raft_consensus.cc:348] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "099a625a94ff4147881504b477a5305e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43375 } }
I20260929 11:01:44.372768 8413 raft_consensus.cc:374] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:01:44.372792 8413 raft_consensus.cc:729] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 099a625a94ff4147881504b477a5305e, State: Initialized, Role: FOLLOWER
I20260929 11:01:44.372974 8413 consensus_queue.cc:260] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "099a625a94ff4147881504b477a5305e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43375 } }
I20260929 11:01:44.373064 8413 raft_consensus.cc:388] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 11:01:44.373101 8413 raft_consensus.cc:482] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 11:01:44.373152 8413 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:01:44.374248 8413 raft_consensus.cc:504] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "099a625a94ff4147881504b477a5305e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43375 } }
I20260929 11:01:44.374336 8413 leader_election.cc:302] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 099a625a94ff4147881504b477a5305e; no voters:
I20260929 11:01:44.374576 8413 leader_election.cc:288] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 11:01:44.374598 8418 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:01:44.374703 8418 raft_consensus.cc:686] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [term 1 LEADER]: Becoming Leader. State: Replica: 099a625a94ff4147881504b477a5305e, State: Running, Role: LEADER
I20260929 11:01:44.374840 8413 sys_catalog.cc:564] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:01:44.374833 8418 consensus_queue.cc:237] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "099a625a94ff4147881504b477a5305e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43375 } }
I20260929 11:01:44.376755 8420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "099a625a94ff4147881504b477a5305e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "099a625a94ff4147881504b477a5305e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43375 } } }
I20260929 11:01:44.376972 8419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 099a625a94ff4147881504b477a5305e. Latest consensus state: current_term: 1 leader_uuid: "099a625a94ff4147881504b477a5305e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "099a625a94ff4147881504b477a5305e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43375 } } }
I20260929 11:01:44.377032 8420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [sys.catalog]: This master's current role is: LEADER
W20260929 11:01:44.377100 8432 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260929 11:01:44.377048 8419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [sys.catalog]: This master's current role is: LEADER
W20260929 11:01:44.377169 8432 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 11:01:44.377799 8434 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:01:44.377976 8434 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 29 11:01:44 dist-test-slave-2lzq krb5kdc[8332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:01:44.379691 8434 catalog_manager.cc:1347] Generated new cluster ID: 95defd70f03a4de6b5c5db20a33104a1
I20260929 11:01:44.379737 8434 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 11:01:44.383224 8417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:44.371186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35758 (local address 127.26.120.254:43375)
0929 11:01:44.371356 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:44.371366 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:44.371401 (+ 35us) server_negotiation.cc:407] Connection header received
0929 11:01:44.372628 (+ 1227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:44.372636 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:44.372690 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:44.372871 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:01:44.373671 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.374860 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.375732 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.376202 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.376356 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:44.379300 (+ 2944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:01:44.379304 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:01:44.379305 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:01:44.379317 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:01:44.381268 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:44.381651 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:44.381654 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:44.381655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:44.381778 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:44.381956 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:44.381957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:44.381957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:44.382279 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:01:44.382431 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:44.382889 (+ 458us) server_negotiation.cc:299] Negotiation successful
0929 11:01:44.383027 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":93,"threads_started":1}
I20260929 11:01:44.385700 8434 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 11:01:44.386436 8434 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:01:44.391625 8434 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e: Generated new TSK 0
I20260929 11:01:44.391815 8434 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.120.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab.
Entry for principal kudu/127.26.120.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab.
I20260929 11:01:44.420668 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.193:0
--local_ip_for_outbound_sockets=127.26.120.193
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:43375
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:33997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:01:44.501287 8439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:01:44.501520 8439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:01:44.501560 8439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:01:44.502790 8439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:01:44.502853 8439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:01:44.502895 8439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.193
I20260929 11:01:44.504333 8439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:33997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:43375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.193
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:01:44.504598 8439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:01:44.504828 8439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:01:44.510674 8447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:01:44.510674 8444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:01:44.510712 8445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:01:44.510743 8439 server_base.cc:1034] running on GCE node
I20260929 11:01:44.511207 8439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:01:44.511448 8439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:01:44.512614 8439 hybrid_clock.cc:648] HybridClock initialized: now 1790679704512604 us; error 37 us; skew 500 ppm
Sep 29 11:01:44 dist-test-slave-2lzq krb5kdc[8332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679704, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:01:44.515276 8439 init.cc:377] Logged in from keytab as kudu/127.26.120.193@KRBTEST.COM (short username kudu)
I20260929 11:01:44.515750 8439 webserver.cc:460] Webserver started at http://127.26.120.193:46413/ using document root <none> and password file <none>
I20260929 11:01:44.515954 8439 fs_manager.cc:359] Metadata directory not provided
I20260929 11:01:44.515993 8439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:01:44.516080 8439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:01:44.517077 8439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/data/instance:
uuid: "3458dd5df95c4051b0962ec6aa134b41"
format_stamp: "Formatted at 2026-09-29 11:01:44 on dist-test-slave-2lzq"
server_key: "d1b56b76257e6d840e63b9488ef5a1d8"
server_key_iv: "a2d7e63b12db236689e4fdf07978d602"
server_key_version: "encryptionkey@0"
I20260929 11:01:44.517388 8439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance:
uuid: "3458dd5df95c4051b0962ec6aa134b41"
format_stamp: "Formatted at 2026-09-29 11:01:44 on dist-test-slave-2lzq"
server_key: "d1b56b76257e6d840e63b9488ef5a1d8"
server_key_iv: "a2d7e63b12db236689e4fdf07978d602"
server_key_version: "encryptionkey@0"
I20260929 11:01:44.518746 8439 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 11:01:44.519475 8454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:44.519676 8439 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 11:01:44.519766 8439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/wal
uuid: "3458dd5df95c4051b0962ec6aa134b41"
format_stamp: "Formatted at 2026-09-29 11:01:44 on dist-test-slave-2lzq"
server_key: "d1b56b76257e6d840e63b9488ef5a1d8"
server_key_iv: "a2d7e63b12db236689e4fdf07978d602"
server_key_version: "encryptionkey@0"
I20260929 11:01:44.519824 8439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:01:44.545919 8439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:01:44.548305 8439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:01:44.548457 8439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:01:44.548715 8439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:01:44.549038 8439 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:01:44.549075 8439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:44.549098 8439 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:01:44.549144 8439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:44.554965 8439 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.193:37655
I20260929 11:01:44.555040 8567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.193:37655 every 8 connection(s)
I20260929 11:01:44.555384 8439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
I20260929 11:01:44.556165 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 8439
I20260929 11:01:44.556283 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance
I20260929 11:01:44.556550 27107 external_mini_cluster.cc:1428] Setting key fb9f415c0f5447ae24499362a4df8bf2
Sep 29 11:01:44 dist-test-slave-2lzq krb5kdc[8332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679704, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.120.194@KRBTEST.COM; defaulting to no policy
I20260929 11:01:44.565727 8417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:44.556229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:46197 (local address 127.26.120.254:43375)
0929 11:01:44.556752 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:44.556754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:44.556768 (+ 14us) server_negotiation.cc:407] Connection header received
0929 11:01:44.557144 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:44.557146 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:44.557161 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:44.557214 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:01:44.558303 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.558820 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.559867 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.560093 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.560183 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:44.562779 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:01:44.562780 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:01:44.562780 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:01:44.562789 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:01:44.564219 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:44.564734 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:44.564734 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:44.564735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:44.564786 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:44.565046 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:44.565046 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:44.565047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:44.565181 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:01:44.565248 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:44.565544 (+ 296us) server_negotiation.cc:299] Negotiation successful
0929 11:01:44.565632 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490}
I20260929 11:01:44.565949 8570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:44.556236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.193:46197)
0929 11:01:44.556467 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:44.556522 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:44.557029 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:44.557318 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:44.557322 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:44.557920 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:01:44.558205 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:44.558212 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.558936 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.558937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:44.559770 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:44.559772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.560194 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.560195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:44.560331 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:44.560888 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:01:44.560897 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:01:44.562678 (+ 1781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:01:44.564319 (+ 1641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:44.564327 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:44.564329 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:44.564652 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:44.564851 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:44.564852 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:44.564852 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:44.564984 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:44.565241 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:01:44.565248 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:01:44.565402 (+ 154us) client_negotiation.cc:769] Sending connection context
0929 11:01:44.565517 (+ 115us) client_negotiation.cc:240] Negotiation successful
0929 11:01:44.565679 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":117,"threads_started":1}
I20260929 11:01:44.566505 8568 heartbeater.cc:347] Connected to a master server at 127.26.120.254:43375
I20260929 11:01:44.566618 8568 heartbeater.cc:464] Registering TS with master...
I20260929 11:01:44.566805 8568 heartbeater.cc:511] Master 127.26.120.254:43375 requested a full tablet report, sending...
I20260929 11:01:44.567384 8377 ts_manager.cc:194] Registered new tserver with Master: 3458dd5df95c4051b0962ec6aa134b41 (127.26.120.193:37655)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.194@KRBTEST.COM" created.
I20260929 11:01:44.568164 8377 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.193@KRBTEST.COM'} at 127.26.120.193:46197
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab.
Entry for principal kudu/127.26.120.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab.
I20260929 11:01:44.580856 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.194:0
--local_ip_for_outbound_sockets=127.26.120.194
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:43375
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:33997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:01:44.660800 8573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:01:44.661046 8573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:01:44.661082 8573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:01:44.662216 8573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:01:44.662285 8573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:01:44.662331 8573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.194
I20260929 11:01:44.663769 8573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:33997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:43375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.194
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:01:44.664042 8573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:01:44.664276 8573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:01:44.669420 8578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:01:44.669410 8579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:01:44.669741 8573 server_base.cc:1034] running on GCE node
W20260929 11:01:44.669409 8581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:01:44.670030 8573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:01:44.670321 8573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:01:44.671483 8573 hybrid_clock.cc:648] HybridClock initialized: now 1790679704671472 us; error 26 us; skew 500 ppm
Sep 29 11:01:44 dist-test-slave-2lzq krb5kdc[8332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679704, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:01:44.673859 8573 init.cc:377] Logged in from keytab as kudu/127.26.120.194@KRBTEST.COM (short username kudu)
I20260929 11:01:44.674356 8573 webserver.cc:460] Webserver started at http://127.26.120.194:36835/ using document root <none> and password file <none>
I20260929 11:01:44.674607 8573 fs_manager.cc:359] Metadata directory not provided
I20260929 11:01:44.674676 8573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:01:44.674829 8573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:01:44.676074 8573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/data/instance:
uuid: "3d903e30c836441eab86034bebe7421d"
format_stamp: "Formatted at 2026-09-29 11:01:44 on dist-test-slave-2lzq"
server_key: "dbf51f0d0dc37f38ec6b6f2742ae6e7b"
server_key_iv: "d169e87d10e0dd04f2f28eb3c05337ae"
server_key_version: "encryptionkey@0"
I20260929 11:01:44.676452 8573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance:
uuid: "3d903e30c836441eab86034bebe7421d"
format_stamp: "Formatted at 2026-09-29 11:01:44 on dist-test-slave-2lzq"
server_key: "dbf51f0d0dc37f38ec6b6f2742ae6e7b"
server_key_iv: "d169e87d10e0dd04f2f28eb3c05337ae"
server_key_version: "encryptionkey@0"
I20260929 11:01:44.677790 8573 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 11:01:44.678602 8588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:44.678794 8573 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:44.678890 8573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/wal
uuid: "3d903e30c836441eab86034bebe7421d"
format_stamp: "Formatted at 2026-09-29 11:01:44 on dist-test-slave-2lzq"
server_key: "dbf51f0d0dc37f38ec6b6f2742ae6e7b"
server_key_iv: "d169e87d10e0dd04f2f28eb3c05337ae"
server_key_version: "encryptionkey@0"
I20260929 11:01:44.678977 8573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:01:44.687062 8573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:01:44.688928 8573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:01:44.689095 8573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:01:44.689325 8573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:01:44.689632 8573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:01:44.689687 8573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:44.689728 8573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:01:44.689754 8573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:44.696851 8573 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.194:35157
I20260929 11:01:44.696977 8701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.194:35157 every 8 connection(s)
I20260929 11:01:44.697258 8573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
Sep 29 11:01:44 dist-test-slave-2lzq krb5kdc[8332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679704, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:01:44.705574 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 8573
I20260929 11:01:44.705677 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance
I20260929 11:01:44.705870 27107 external_mini_cluster.cc:1428] Setting key f1df352727e95512c641450d68844451
I20260929 11:01:44.708566 8417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:44.698212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:55323 (local address 127.26.120.254:43375)
0929 11:01:44.698298 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:44.698300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:44.698531 (+ 231us) server_negotiation.cc:407] Connection header received
0929 11:01:44.699074 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:44.699075 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:44.699089 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:44.699136 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:01:44.700412 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.701139 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.702633 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.703010 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.703078 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:44.705346 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:01:44.705348 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:01:44.705348 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:01:44.705355 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:01:44.707154 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:44.707550 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:44.707551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:44.707551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:44.707617 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:44.707853 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:44.707854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:44.707854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:44.708012 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:01:44.708081 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:44.708382 (+ 301us) server_negotiation.cc:299] Negotiation successful
0929 11:01:44.708474 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 11:01:44.708781 8704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:44.698244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.194:55323)
0929 11:01:44.698398 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:44.698438 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:44.698956 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:44.699204 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:44.699208 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:44.699975 (+ 767us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:01:44.700253 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:44.700261 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.701251 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.701252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:44.702493 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:44.702496 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.703157 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.703158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:44.703335 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:44.703903 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:01:44.703916 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:01:44.705258 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:01:44.707239 (+ 1981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:44.707246 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:44.707248 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:44.707471 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:44.707685 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:44.707686 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:44.707686 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:44.707788 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:44.708075 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:01:44.708080 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:01:44.708235 (+ 155us) client_negotiation.cc:769] Sending connection context
0929 11:01:44.708356 (+ 121us) client_negotiation.cc:240] Negotiation successful
0929 11:01:44.708536 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":86,"threads_started":1}
I20260929 11:01:44.709323 8702 heartbeater.cc:347] Connected to a master server at 127.26.120.254:43375
I20260929 11:01:44.709420 8702 heartbeater.cc:464] Registering TS with master...
I20260929 11:01:44.709628 8702 heartbeater.cc:511] Master 127.26.120.254:43375 requested a full tablet report, sending...
I20260929 11:01:44.710005 8377 ts_manager.cc:194] Registered new tserver with Master: 3d903e30c836441eab86034bebe7421d (127.26.120.194:35157)
I20260929 11:01:44.710419 8377 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.194@KRBTEST.COM'} at 127.26.120.194:55323
WARNING: no policy specified for kudu/127.26.120.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab.
Entry for principal kudu/127.26.120.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab.
I20260929 11:01:44.732508 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.195:0
--local_ip_for_outbound_sockets=127.26.120.195
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:43375
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:33997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:01:44.813902 8707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:01:44.814128 8707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:01:44.814152 8707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:01:44.815223 8707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:01:44.815254 8707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:01:44.815279 8707 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.195
I20260929 11:01:44.816642 8707 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:33997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:43375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.195
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:01:44.816828 8707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:01:44.817065 8707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:01:44.822150 8713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:01:44.822151 8712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:01:44.822350 8707 server_base.cc:1034] running on GCE node
W20260929 11:01:44.822156 8715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:01:44.822654 8707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:01:44.822914 8707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:01:44.824080 8707 hybrid_clock.cc:648] HybridClock initialized: now 1790679704824054 us; error 36 us; skew 500 ppm
Sep 29 11:01:44 dist-test-slave-2lzq krb5kdc[8332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679704, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:01:44.826795 8707 init.cc:377] Logged in from keytab as kudu/127.26.120.195@KRBTEST.COM (short username kudu)
I20260929 11:01:44.827281 8707 webserver.cc:460] Webserver started at http://127.26.120.195:46053/ using document root <none> and password file <none>
I20260929 11:01:44.827526 8707 fs_manager.cc:359] Metadata directory not provided
I20260929 11:01:44.827594 8707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:01:44.827739 8707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:01:44.828821 8707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/data/instance:
uuid: "d6b40d28ccab483f9cda4f705e51e737"
format_stamp: "Formatted at 2026-09-29 11:01:44 on dist-test-slave-2lzq"
server_key: "6c622702ddcc679d35bd0e0ac86ab89c"
server_key_iv: "0d84a0d2a688f8ef0207922b61ce0b08"
server_key_version: "encryptionkey@0"
I20260929 11:01:44.829177 8707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance:
uuid: "d6b40d28ccab483f9cda4f705e51e737"
format_stamp: "Formatted at 2026-09-29 11:01:44 on dist-test-slave-2lzq"
server_key: "6c622702ddcc679d35bd0e0ac86ab89c"
server_key_iv: "0d84a0d2a688f8ef0207922b61ce0b08"
server_key_version: "encryptionkey@0"
I20260929 11:01:44.830629 8707 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 11:01:44.831351 8722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:44.831524 8707 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 11:01:44.831635 8707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/wal
uuid: "d6b40d28ccab483f9cda4f705e51e737"
format_stamp: "Formatted at 2026-09-29 11:01:44 on dist-test-slave-2lzq"
server_key: "6c622702ddcc679d35bd0e0ac86ab89c"
server_key_iv: "0d84a0d2a688f8ef0207922b61ce0b08"
server_key_version: "encryptionkey@0"
I20260929 11:01:44.831708 8707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:01:44.847049 8707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:01:44.849412 8707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:01:44.849581 8707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:01:44.849822 8707 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:01:44.850147 8707 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:01:44.850201 8707 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:44.850241 8707 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:01:44.850271 8707 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:44.856124 8707 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.195:45339
I20260929 11:01:44.856201 8835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.195:45339 every 8 connection(s)
I20260929 11:01:44.856534 8707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
I20260929 11:01:44.857856 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 8707
I20260929 11:01:44.857961 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance
I20260929 11:01:44.858156 27107 external_mini_cluster.cc:1428] Setting key 46480d28f7e64db71f972420e24092b6
Sep 29 11:01:44 dist-test-slave-2lzq krb5kdc[8332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679704, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:01:44.866215 8417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:44.857245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:32991 (local address 127.26.120.254:43375)
0929 11:01:44.857320 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:44.857322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:44.857558 (+ 236us) server_negotiation.cc:407] Connection header received
0929 11:01:44.858180 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:44.858182 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:44.858195 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:44.858261 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:01:44.859436 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.859951 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.860976 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.861190 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.861275 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:44.863450 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:01:44.863452 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:01:44.863452 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:01:44.863460 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:01:44.864725 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:44.865117 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:44.865118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:44.865118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:44.865168 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:44.865418 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:44.865419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:44.865419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:44.865525 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:01:44.865579 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:44.866009 (+ 430us) server_negotiation.cc:299] Negotiation successful
0929 11:01:44.866150 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 11:01:44.866335 8838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:44.857239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.195:32991)
0929 11:01:44.857436 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:44.857472 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:44.858032 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:44.858350 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:44.858353 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:44.859136 (+ 783us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:01:44.859333 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:44.859338 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.860051 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.860052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:44.860870 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:44.860873 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.861286 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.861286 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:44.861424 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:44.861883 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:01:44.861891 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:01:44.863348 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:01:44.864804 (+ 1456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:44.864810 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:44.864811 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:44.865049 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:44.865225 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:44.865225 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:44.865225 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:44.865324 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:44.865617 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:01:44.865626 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:01:44.865805 (+ 179us) client_negotiation.cc:769] Sending connection context
0929 11:01:44.865940 (+ 135us) client_negotiation.cc:240] Negotiation successful
0929 11:01:44.866140 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":91,"threads_started":1}
I20260929 11:01:44.866891 8836 heartbeater.cc:347] Connected to a master server at 127.26.120.254:43375
I20260929 11:01:44.866998 8836 heartbeater.cc:464] Registering TS with master...
I20260929 11:01:44.867182 8836 heartbeater.cc:511] Master 127.26.120.254:43375 requested a full tablet report, sending...
I20260929 11:01:44.867574 8377 ts_manager.cc:194] Registered new tserver with Master: d6b40d28ccab483f9cda4f705e51e737 (127.26.120.195:45339)
I20260929 11:01:44.867941 8377 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.195@KRBTEST.COM'} at 127.26.120.195:32991
I20260929 11:01:44.869768 27107 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 11:01:44.877233 8417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:44.871185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35766 (local address 127.26.120.254:43375)
0929 11:01:44.871245 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:44.871247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:44.871388 (+ 141us) server_negotiation.cc:407] Connection header received
0929 11:01:44.871426 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:44.871427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:44.871443 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:44.871485 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:01:44.872014 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.872470 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.873288 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.873463 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.873511 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:44.874732 (+ 1221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:01:44.874734 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:01:44.874734 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:01:44.874739 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:01:44.876196 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:44.876508 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:44.876509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:44.876509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:44.876570 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:44.876743 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:44.876744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:44.876744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:44.876931 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:01:44.877008 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:44.877098 (+ 90us) server_negotiation.cc:299] Negotiation successful
0929 11:01:44.877152 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260929 11:01:44.878829 8377 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35766:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 11:01:44.880339 8377 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 11:01:44.886894 8848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:44.882645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37680 (local address 127.26.120.194:35157)
0929 11:01:44.883178 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:44.883182 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:44.883193 (+ 11us) server_negotiation.cc:407] Connection header received
0929 11:01:44.883232 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:44.883234 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:44.883250 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:44.883326 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:01:44.883467 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.884251 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.885872 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.886453 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.886551 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:44.886607 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:44.886773 (+ 166us) server_negotiation.cc:299] Negotiation successful
0929 11:01:44.886812 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":73,"threads_started":1}
I20260929 11:01:44.886895 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:44.882415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:35157 (local address 127.0.0.1:37680)
0929 11:01:44.882647 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:44.882686 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:44.882753 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:44.883325 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:44.883326 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:44.883328 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:01:44.883404 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:44.883406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.884353 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.884354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:44.885788 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:44.885790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.886551 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.886552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:44.886643 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:44.886656 (+ 13us) client_negotiation.cc:769] Sending connection context
0929 11:01:44.886756 (+ 100us) client_negotiation.cc:240] Negotiation successful
0929 11:01:44.886812 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":59,"thread_start_us":55,"threads_started":1}
I20260929 11:01:44.887060 8849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:44.882651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54502 (local address 127.26.120.195:45339)
0929 11:01:44.883453 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:44.883457 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:44.883466 (+ 9us) server_negotiation.cc:407] Connection header received
0929 11:01:44.883499 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:44.883501 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:44.883515 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:44.883592 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:01:44.883715 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.884652 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.885771 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.886519 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.886595 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:44.886645 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:44.886935 (+ 290us) server_negotiation.cc:299] Negotiation successful
0929 11:01:44.886974 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":769,"thread_start_us":77,"threads_started":1}
I20260929 11:01:44.887236 8846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:44.882416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:37655 (local address 127.0.0.1:60188)
0929 11:01:44.882732 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:44.882744 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:44.882800 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:44.883022 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:44.883024 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:44.883026 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:01:44.883094 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:44.883096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.884426 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.884426 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:44.885756 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:44.885758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.886684 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.886686 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:44.886776 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:44.886779 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 11:01:44.886824 (+ 45us) client_negotiation.cc:240] Negotiation successful
0929 11:01:44.886871 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":76,"thread_start_us":54,"threads_started":1}
I20260929 11:01:44.887248 8845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:44.882468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:45339 (local address 127.0.0.1:54502)
0929 11:01:44.882873 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:44.882882 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:44.882917 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:44.883605 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:44.883605 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:44.883608 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:01:44.883650 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:44.883652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.884758 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.884759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:44.885682 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:44.885685 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.886686 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.886686 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:44.886768 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:44.886773 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 11:01:44.886824 (+ 51us) client_negotiation.cc:240] Negotiation successful
0929 11:01:44.886872 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"mutex_wait_us":38,"thread_start_us":34,"threads_started":1}
I20260929 11:01:44.887418 8847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:44.882546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60188 (local address 127.26.120.193:37655)
0929 11:01:44.882843 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:44.882848 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:44.882861 (+ 13us) server_negotiation.cc:407] Connection header received
0929 11:01:44.882906 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:44.882908 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:44.882926 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:44.883019 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:01:44.883184 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.884304 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.885831 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.886531 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.886595 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:44.886645 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:44.886918 (+ 273us) server_negotiation.cc:299] Negotiation successful
0929 11:01:44.886959 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":92,"threads_started":1}
I20260929 11:01:44.887806 8636 tablet_service.cc:1467] Processing CreateTablet for tablet 061863abbd6642719baf4c53c2311718 (DEFAULT_TABLE table=test-table [id=36299631cd8d4abf8ea1123bf612e76a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:01:44.888001 8502 tablet_service.cc:1467] Processing CreateTablet for tablet 061863abbd6642719baf4c53c2311718 (DEFAULT_TABLE table=test-table [id=36299631cd8d4abf8ea1123bf612e76a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:01:44.888001 8770 tablet_service.cc:1467] Processing CreateTablet for tablet 061863abbd6642719baf4c53c2311718 (DEFAULT_TABLE table=test-table [id=36299631cd8d4abf8ea1123bf612e76a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:01:44.888175 8636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 061863abbd6642719baf4c53c2311718. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:01:44.888314 8502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 061863abbd6642719baf4c53c2311718. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:01:44.888314 8770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 061863abbd6642719baf4c53c2311718. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:01:44.890921 8850 tablet_bootstrap.cc:492] T 061863abbd6642719baf4c53c2311718 P 3d903e30c836441eab86034bebe7421d: Bootstrap starting.
I20260929 11:01:44.891527 8851 tablet_bootstrap.cc:492] T 061863abbd6642719baf4c53c2311718 P 3458dd5df95c4051b0962ec6aa134b41: Bootstrap starting.
I20260929 11:01:44.891656 8850 tablet_bootstrap.cc:654] T 061863abbd6642719baf4c53c2311718 P 3d903e30c836441eab86034bebe7421d: Neither blocks nor log segments found. Creating new log.
I20260929 11:01:44.892218 8851 tablet_bootstrap.cc:654] T 061863abbd6642719baf4c53c2311718 P 3458dd5df95c4051b0962ec6aa134b41: Neither blocks nor log segments found. Creating new log.
I20260929 11:01:44.892297 8850 log.cc:824] T 061863abbd6642719baf4c53c2311718 P 3d903e30c836441eab86034bebe7421d: Log is configured to *not* fsync() on all Append() calls
I20260929 11:01:44.892812 8851 log.cc:824] T 061863abbd6642719baf4c53c2311718 P 3458dd5df95c4051b0962ec6aa134b41: Log is configured to *not* fsync() on all Append() calls
I20260929 11:01:44.893157 8852 tablet_bootstrap.cc:492] T 061863abbd6642719baf4c53c2311718 P d6b40d28ccab483f9cda4f705e51e737: Bootstrap starting.
I20260929 11:01:44.893270 8850 tablet_bootstrap.cc:492] T 061863abbd6642719baf4c53c2311718 P 3d903e30c836441eab86034bebe7421d: No bootstrap required, opened a new log
I20260929 11:01:44.893328 8850 ts_tablet_manager.cc:1397] T 061863abbd6642719baf4c53c2311718 P 3d903e30c836441eab86034bebe7421d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 11:01:44.893536 8851 tablet_bootstrap.cc:492] T 061863abbd6642719baf4c53c2311718 P 3458dd5df95c4051b0962ec6aa134b41: No bootstrap required, opened a new log
I20260929 11:01:44.893618 8851 ts_tablet_manager.cc:1397] T 061863abbd6642719baf4c53c2311718 P 3458dd5df95c4051b0962ec6aa134b41: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 11:01:44.893775 8852 tablet_bootstrap.cc:654] T 061863abbd6642719baf4c53c2311718 P d6b40d28ccab483f9cda4f705e51e737: Neither blocks nor log segments found. Creating new log.
I20260929 11:01:44.894400 8852 log.cc:824] T 061863abbd6642719baf4c53c2311718 P d6b40d28ccab483f9cda4f705e51e737: Log is configured to *not* fsync() on all Append() calls
I20260929 11:01:44.895174 8852 tablet_bootstrap.cc:492] T 061863abbd6642719baf4c53c2311718 P d6b40d28ccab483f9cda4f705e51e737: No bootstrap required, opened a new log
I20260929 11:01:44.895246 8852 ts_tablet_manager.cc:1397] T 061863abbd6642719baf4c53c2311718 P d6b40d28ccab483f9cda4f705e51e737: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 11:01:44.895401 8851 raft_consensus.cc:348] T 061863abbd6642719baf4c53c2311718 P 3458dd5df95c4051b0962ec6aa134b41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d903e30c836441eab86034bebe7421d" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35157 } } peers { permanent_uuid: "3458dd5df95c4051b0962ec6aa134b41" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37655 } } peers { permanent_uuid: "d6b40d28ccab483f9cda4f705e51e737" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45339 } }
I20260929 11:01:44.895550 8851 raft_consensus.cc:374] T 061863abbd6642719baf4c53c2311718 P 3458dd5df95c4051b0962ec6aa134b41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:01:44.895478 8850 raft_consensus.cc:348] T 061863abbd6642719baf4c53c2311718 P 3d903e30c836441eab86034bebe7421d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d903e30c836441eab86034bebe7421d" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35157 } } peers { permanent_uuid: "3458dd5df95c4051b0962ec6aa134b41" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37655 } } peers { permanent_uuid: "d6b40d28ccab483f9cda4f705e51e737" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45339 } }
I20260929 11:01:44.895593 8851 raft_consensus.cc:729] T 061863abbd6642719baf4c53c2311718 P 3458dd5df95c4051b0962ec6aa134b41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3458dd5df95c4051b0962ec6aa134b41, State: Initialized, Role: FOLLOWER
I20260929 11:01:44.895622 8850 raft_consensus.cc:374] T 061863abbd6642719baf4c53c2311718 P 3d903e30c836441eab86034bebe7421d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:01:44.895661 8850 raft_consensus.cc:729] T 061863abbd6642719baf4c53c2311718 P 3d903e30c836441eab86034bebe7421d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d903e30c836441eab86034bebe7421d, State: Initialized, Role: FOLLOWER
I20260929 11:01:44.895706 8851 consensus_queue.cc:260] T 061863abbd6642719baf4c53c2311718 P 3458dd5df95c4051b0962ec6aa134b41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d903e30c836441eab86034bebe7421d" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35157 } } peers { permanent_uuid: "3458dd5df95c4051b0962ec6aa134b41" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37655 } } peers { permanent_uuid: "d6b40d28ccab483f9cda4f705e51e737" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45339 } }
I20260929 11:01:44.895839 8850 consensus_queue.cc:260] T 061863abbd6642719baf4c53c2311718 P 3d903e30c836441eab86034bebe7421d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d903e30c836441eab86034bebe7421d" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35157 } } peers { permanent_uuid: "3458dd5df95c4051b0962ec6aa134b41" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37655 } } peers { permanent_uuid: "d6b40d28ccab483f9cda4f705e51e737" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45339 } }
I20260929 11:01:44.895977 8851 ts_tablet_manager.cc:1428] T 061863abbd6642719baf4c53c2311718 P 3458dd5df95c4051b0962ec6aa134b41: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260929 11:01:44.896013 8568 heartbeater.cc:503] Master 127.26.120.254:43375 was elected leader, sending a full tablet report...
I20260929 11:01:44.896116 8850 ts_tablet_manager.cc:1428] T 061863abbd6642719baf4c53c2311718 P 3d903e30c836441eab86034bebe7421d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 11:01:44.896255 8702 heartbeater.cc:503] Master 127.26.120.254:43375 was elected leader, sending a full tablet report...
I20260929 11:01:44.897281 8852 raft_consensus.cc:348] T 061863abbd6642719baf4c53c2311718 P d6b40d28ccab483f9cda4f705e51e737 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d903e30c836441eab86034bebe7421d" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35157 } } peers { permanent_uuid: "3458dd5df95c4051b0962ec6aa134b41" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37655 } } peers { permanent_uuid: "d6b40d28ccab483f9cda4f705e51e737" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45339 } }
I20260929 11:01:44.897411 8852 raft_consensus.cc:374] T 061863abbd6642719baf4c53c2311718 P d6b40d28ccab483f9cda4f705e51e737 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:01:44.897433 8852 raft_consensus.cc:729] T 061863abbd6642719baf4c53c2311718 P d6b40d28ccab483f9cda4f705e51e737 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6b40d28ccab483f9cda4f705e51e737, State: Initialized, Role: FOLLOWER
I20260929 11:01:44.897553 8852 consensus_queue.cc:260] T 061863abbd6642719baf4c53c2311718 P d6b40d28ccab483f9cda4f705e51e737 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d903e30c836441eab86034bebe7421d" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35157 } } peers { permanent_uuid: "3458dd5df95c4051b0962ec6aa134b41" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37655 } } peers { permanent_uuid: "d6b40d28ccab483f9cda4f705e51e737" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45339 } }
I20260929 11:01:44.897833 8852 ts_tablet_manager.cc:1428] T 061863abbd6642719baf4c53c2311718 P d6b40d28ccab483f9cda4f705e51e737: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 11:01:44.897857 8836 heartbeater.cc:503] Master 127.26.120.254:43375 was elected leader, sending a full tablet report...
I20260929 11:01:44.907088 8857 raft_consensus.cc:482] T 061863abbd6642719baf4c53c2311718 P 3d903e30c836441eab86034bebe7421d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 11:01:44.907166 8857 raft_consensus.cc:504] T 061863abbd6642719baf4c53c2311718 P 3d903e30c836441eab86034bebe7421d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d903e30c836441eab86034bebe7421d" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35157 } } peers { permanent_uuid: "3458dd5df95c4051b0962ec6aa134b41" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37655 } } peers { permanent_uuid: "d6b40d28ccab483f9cda4f705e51e737" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45339 } }
I20260929 11:01:44.907457 8857 leader_election.cc:288] T 061863abbd6642719baf4c53c2311718 P 3d903e30c836441eab86034bebe7421d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3458dd5df95c4051b0962ec6aa134b41 (127.26.120.193:37655), d6b40d28ccab483f9cda4f705e51e737 (127.26.120.195:45339)
I20260929 11:01:44.911320 8847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:44.907740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:39255 (local address 127.26.120.193:37655)
0929 11:01:44.907779 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:44.907780 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:44.907831 (+ 51us) server_negotiation.cc:407] Connection header received
0929 11:01:44.907878 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:44.907879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:44.907890 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:44.907942 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:01:44.908101 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.908837 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.910389 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.910963 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.911034 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:44.911042 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:44.911218 (+ 176us) server_negotiation.cc:299] Negotiation successful
0929 11:01:44.911254 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 11:01:44.911320 8849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:44.907656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:37129 (local address 127.26.120.195:45339)
0929 11:01:44.907716 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:44.907718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:44.907728 (+ 10us) server_negotiation.cc:407] Connection header received
0929 11:01:44.907836 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:44.907837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:44.907847 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:44.907912 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:01:44.908101 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.908849 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.910394 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.910956 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.911033 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:44.911042 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:44.911218 (+ 176us) server_negotiation.cc:299] Negotiation successful
0929 11:01:44.911254 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 11:01:44.911336 8704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:44.907596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:45339 (local address 127.26.120.194:37129)
0929 11:01:44.907662 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:44.907675 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:44.907720 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:44.907922 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:44.907923 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:44.907925 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:01:44.908032 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:44.908034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.908931 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.908932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:44.910302 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:44.910305 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.911047 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.911048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:44.911128 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:44.911132 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 11:01:44.911190 (+ 58us) client_negotiation.cc:240] Negotiation successful
0929 11:01:44.911244 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":55}
I20260929 11:01:44.911343 8859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:44.907597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:37655 (local address 127.26.120.194:39255)
0929 11:01:44.907764 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:44.907772 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:44.907824 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:44.907940 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:44.907941 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:44.907942 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:01:44.908022 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:44.908024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.908931 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.908932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:44.910302 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:44.910305 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.911047 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.911048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:44.911128 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:44.911132 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 11:01:44.911190 (+ 58us) client_negotiation.cc:240] Negotiation successful
0929 11:01:44.911242 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":117,"thread_start_us":75,"threads_started":1}
I20260929 11:01:44.911691 8790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "061863abbd6642719baf4c53c2311718" candidate_uuid: "3d903e30c836441eab86034bebe7421d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6b40d28ccab483f9cda4f705e51e737" is_pre_election: true
I20260929 11:01:44.911849 8790 raft_consensus.cc:2454] T 061863abbd6642719baf4c53c2311718 P d6b40d28ccab483f9cda4f705e51e737 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d903e30c836441eab86034bebe7421d in term 0.
I20260929 11:01:44.911852 8522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "061863abbd6642719baf4c53c2311718" candidate_uuid: "3d903e30c836441eab86034bebe7421d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3458dd5df95c4051b0962ec6aa134b41" is_pre_election: true
I20260929 11:01:44.912019 8522 raft_consensus.cc:2454] T 061863abbd6642719baf4c53c2311718 P 3458dd5df95c4051b0962ec6aa134b41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d903e30c836441eab86034bebe7421d in term 0.
I20260929 11:01:44.912132 8591 leader_election.cc:302] T 061863abbd6642719baf4c53c2311718 P 3d903e30c836441eab86034bebe7421d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d903e30c836441eab86034bebe7421d, d6b40d28ccab483f9cda4f705e51e737; no voters:
I20260929 11:01:44.912261 8857 raft_consensus.cc:2781] T 061863abbd6642719baf4c53c2311718 P 3d903e30c836441eab86034bebe7421d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 11:01:44.912329 8857 raft_consensus.cc:482] T 061863abbd6642719baf4c53c2311718 P 3d903e30c836441eab86034bebe7421d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 11:01:44.912364 8857 raft_consensus.cc:3037] T 061863abbd6642719baf4c53c2311718 P 3d903e30c836441eab86034bebe7421d [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:01:44.913059 8857 raft_consensus.cc:504] T 061863abbd6642719baf4c53c2311718 P 3d903e30c836441eab86034bebe7421d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d903e30c836441eab86034bebe7421d" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35157 } } peers { permanent_uuid: "3458dd5df95c4051b0962ec6aa134b41" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37655 } } peers { permanent_uuid: "d6b40d28ccab483f9cda4f705e51e737" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45339 } }
I20260929 11:01:44.913203 8857 leader_election.cc:288] T 061863abbd6642719baf4c53c2311718 P 3d903e30c836441eab86034bebe7421d [CANDIDATE]: Term 1 election: Requested vote from peers 3458dd5df95c4051b0962ec6aa134b41 (127.26.120.193:37655), d6b40d28ccab483f9cda4f705e51e737 (127.26.120.195:45339)
I20260929 11:01:44.913357 8522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "061863abbd6642719baf4c53c2311718" candidate_uuid: "3d903e30c836441eab86034bebe7421d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3458dd5df95c4051b0962ec6aa134b41"
I20260929 11:01:44.913360 8790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "061863abbd6642719baf4c53c2311718" candidate_uuid: "3d903e30c836441eab86034bebe7421d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6b40d28ccab483f9cda4f705e51e737"
I20260929 11:01:44.913437 8790 raft_consensus.cc:3037] T 061863abbd6642719baf4c53c2311718 P d6b40d28ccab483f9cda4f705e51e737 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:01:44.913437 8522 raft_consensus.cc:3037] T 061863abbd6642719baf4c53c2311718 P 3458dd5df95c4051b0962ec6aa134b41 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:01:44.914340 8522 raft_consensus.cc:2454] T 061863abbd6642719baf4c53c2311718 P 3458dd5df95c4051b0962ec6aa134b41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d903e30c836441eab86034bebe7421d in term 1.
I20260929 11:01:44.914340 8790 raft_consensus.cc:2454] T 061863abbd6642719baf4c53c2311718 P d6b40d28ccab483f9cda4f705e51e737 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d903e30c836441eab86034bebe7421d in term 1.
I20260929 11:01:44.914520 8589 leader_election.cc:302] T 061863abbd6642719baf4c53c2311718 P 3d903e30c836441eab86034bebe7421d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3458dd5df95c4051b0962ec6aa134b41, 3d903e30c836441eab86034bebe7421d; no voters:
I20260929 11:01:44.914623 8857 raft_consensus.cc:2781] T 061863abbd6642719baf4c53c2311718 P 3d903e30c836441eab86034bebe7421d [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:01:44.914760 8857 raft_consensus.cc:686] T 061863abbd6642719baf4c53c2311718 P 3d903e30c836441eab86034bebe7421d [term 1 LEADER]: Becoming Leader. State: Replica: 3d903e30c836441eab86034bebe7421d, State: Running, Role: LEADER
I20260929 11:01:44.914861 8857 consensus_queue.cc:237] T 061863abbd6642719baf4c53c2311718 P 3d903e30c836441eab86034bebe7421d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d903e30c836441eab86034bebe7421d" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35157 } } peers { permanent_uuid: "3458dd5df95c4051b0962ec6aa134b41" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37655 } } peers { permanent_uuid: "d6b40d28ccab483f9cda4f705e51e737" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45339 } }
I20260929 11:01:44.915665 8375 catalog_manager.cc:5495] T 061863abbd6642719baf4c53c2311718 P 3d903e30c836441eab86034bebe7421d reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d903e30c836441eab86034bebe7421d (127.26.120.194). New cstate: current_term: 1 leader_uuid: "3d903e30c836441eab86034bebe7421d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d903e30c836441eab86034bebe7421d" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3458dd5df95c4051b0962ec6aa134b41" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6b40d28ccab483f9cda4f705e51e737" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45339 } health_report { overall_health: UNKNOWN } } }
I20260929 11:01:44.924613 8848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:44.922135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37692 (local address 127.26.120.194:35157)
0929 11:01:44.922214 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:44.922216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:44.922225 (+ 9us) server_negotiation.cc:407] Connection header received
0929 11:01:44.922303 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:44.922304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:44.922325 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:44.922387 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 11:01:44.922491 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.923025 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.923820 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:44.924039 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:44.924116 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:44.924263 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 11:01:44.924370 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 11:01:44.924466 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:44.924484 (+ 18us) server_negotiation.cc:299] Negotiation successful
0929 11:01:44.924504 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 11:01:44.926417 8790 raft_consensus.cc:1261] T 061863abbd6642719baf4c53c2311718 P d6b40d28ccab483f9cda4f705e51e737 [term 1 FOLLOWER]: Refusing update from remote peer 3d903e30c836441eab86034bebe7421d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 11:01:44.926416 8522 raft_consensus.cc:1261] T 061863abbd6642719baf4c53c2311718 P 3458dd5df95c4051b0962ec6aa134b41 [term 1 FOLLOWER]: Refusing update from remote peer 3d903e30c836441eab86034bebe7421d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 11:01:44.926807 8860 consensus_queue.cc:1035] T 061863abbd6642719baf4c53c2311718 P 3d903e30c836441eab86034bebe7421d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6b40d28ccab483f9cda4f705e51e737" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 11:01:44.926944 8857 consensus_queue.cc:1035] T 061863abbd6642719baf4c53c2311718 P 3d903e30c836441eab86034bebe7421d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3458dd5df95c4051b0962ec6aa134b41" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 11:01:44.929121 8863 mvcc.cc:205] Tried to move back new op lower bound from 7334624071375683584 to 7334624071331979264. Current Snapshot: MvccSnapshot[applied={T|T < 7334624071375683584}]
I20260929 11:01:44.929735 8865 mvcc.cc:205] Tried to move back new op lower bound from 7334624071375683584 to 7334624071331979264. Current Snapshot: MvccSnapshot[applied={T|T < 7334624071375683584 or (T in {7334624071375683584})}]
I20260929 11:01:44.931897 8375 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 11:01:44.932001 8375 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 11:01:44.932767 8375 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e: Sending DeleteTablet for 3 replicas of tablet 061863abbd6642719baf4c53c2311718
I20260929 11:01:44.933087 8636 tablet_service.cc:1514] Processing DeleteTablet for tablet 061863abbd6642719baf4c53c2311718 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:01:44 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:37680
I20260929 11:01:44.933168 8502 tablet_service.cc:1514] Processing DeleteTablet for tablet 061863abbd6642719baf4c53c2311718 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:01:44 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:60188
I20260929 11:01:44.933382 8873 tablet_replica.cc:330] T 061863abbd6642719baf4c53c2311718 P 3d903e30c836441eab86034bebe7421d: stopping tablet replica
I20260929 11:01:44.933393 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 8439
I20260929 11:01:44.933393 8770 tablet_service.cc:1514] Processing DeleteTablet for tablet 061863abbd6642719baf4c53c2311718 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:01:44 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:54502
I20260929 11:01:44.933482 8873 raft_consensus.cc:2229] T 061863abbd6642719baf4c53c2311718 P 3d903e30c836441eab86034bebe7421d [term 1 LEADER]: Raft consensus shutting down.
I20260929 11:01:44.933573 8873 raft_consensus.cc:2258] T 061863abbd6642719baf4c53c2311718 P 3d903e30c836441eab86034bebe7421d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 11:01:44.934439 8873 ts_tablet_manager.cc:1905] T 061863abbd6642719baf4c53c2311718 P 3d903e30c836441eab86034bebe7421d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 11:01:44.936494 8873 ts_tablet_manager.cc:1918] T 061863abbd6642719baf4c53c2311718 P 3d903e30c836441eab86034bebe7421d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 11:01:44.936570 8873 log.cc:1196] T 061863abbd6642719baf4c53c2311718 P 3d903e30c836441eab86034bebe7421d: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/wal/wals/061863abbd6642719baf4c53c2311718
I20260929 11:01:44.936848 8873 ts_tablet_manager.cc:1939] T 061863abbd6642719baf4c53c2311718 P 3d903e30c836441eab86034bebe7421d: Deleting consensus metadata
I20260929 11:01:44.937287 8362 catalog_manager.cc:4841] TS 3d903e30c836441eab86034bebe7421d (127.26.120.194:35157): tablet 061863abbd6642719baf4c53c2311718 (table test-table [id=36299631cd8d4abf8ea1123bf612e76a]) successfully deleted
I20260929 11:01:44.938180 8875 tablet_replica.cc:330] T 061863abbd6642719baf4c53c2311718 P d6b40d28ccab483f9cda4f705e51e737: stopping tablet replica
I20260929 11:01:44.938297 8875 raft_consensus.cc:2229] T 061863abbd6642719baf4c53c2311718 P d6b40d28ccab483f9cda4f705e51e737 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 11:01:44.938408 8875 raft_consensus.cc:2258] T 061863abbd6642719baf4c53c2311718 P d6b40d28ccab483f9cda4f705e51e737 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 11:01:44.939168 8875 ts_tablet_manager.cc:1905] T 061863abbd6642719baf4c53c2311718 P d6b40d28ccab483f9cda4f705e51e737: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260929 11:01:44.940809 8361 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.120.193:37655 (error 108)
I20260929 11:01:44.941045 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 8573
I20260929 11:01:44.941045 8875 ts_tablet_manager.cc:1918] T 061863abbd6642719baf4c53c2311718 P d6b40d28ccab483f9cda4f705e51e737: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 11:01:44.941118 8875 log.cc:1196] T 061863abbd6642719baf4c53c2311718 P d6b40d28ccab483f9cda4f705e51e737: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/wal/wals/061863abbd6642719baf4c53c2311718
I20260929 11:01:44.941385 8875 ts_tablet_manager.cc:1939] T 061863abbd6642719baf4c53c2311718 P d6b40d28ccab483f9cda4f705e51e737: Deleting consensus metadata
I20260929 11:01:44.941810 8363 catalog_manager.cc:4841] TS d6b40d28ccab483f9cda4f705e51e737 (127.26.120.195:45339): tablet 061863abbd6642719baf4c53c2311718 (table test-table [id=36299631cd8d4abf8ea1123bf612e76a]) successfully deleted
I20260929 11:01:44.942467 8845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:44.942338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:37655 (local address 127.0.0.1:60198)
0929 11:01:44.942389 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:44.942428 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.193:37655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260929 11:01:44.942605 8361 catalog_manager.cc:4584] TS 3458dd5df95c4051b0962ec6aa134b41 (127.26.120.193:37655): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 061863abbd6642719baf4c53c2311718: Network error: Client connection negotiation failed: client connection to 127.26.120.193:37655: connect: Connection refused (error 111)
I20260929 11:01:44.946280 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 8707
I20260929 11:01:44.951009 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 8345
I20260929 11:01:44.956375 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:43375
--webserver_interface=127.26.120.254
--webserver_port=37609
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:33997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:43375
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:01:45.036617 8877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:01:45.036861 8877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:01:45.036898 8877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:01:45.038122 8877 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:01:45.038195 8877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:01:45.038219 8877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:01:45.038241 8877 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:01:45.038272 8877 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:01:45.039762 8877 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:33997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:43375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:43375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=37609
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:01:45.040040 8877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:01:45.040277 8877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:01:45.049382 8886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:01:45.049382 8884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:01:45.049417 8877 server_base.cc:1034] running on GCE node
W20260929 11:01:45.049382 8883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:01:45.049871 8877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:01:45.050289 8877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:01:45.051443 8877 hybrid_clock.cc:648] HybridClock initialized: now 1790679705051421 us; error 38 us; skew 500 ppm
Sep 29 11:01:45 dist-test-slave-2lzq krb5kdc[8332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679705, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:01:45.057504 8877 init.cc:377] Logged in from keytab as kudu/127.26.120.254@KRBTEST.COM (short username kudu)
I20260929 11:01:45.058015 8877 webserver.cc:460] Webserver started at http://127.26.120.254:37609/ using document root <none> and password file <none>
I20260929 11:01:45.058284 8877 fs_manager.cc:359] Metadata directory not provided
I20260929 11:01:45.058346 8877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:01:45.059687 8877 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:01:45.060323 8893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:45.060515 8877 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 11:01:45.060609 8877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "099a625a94ff4147881504b477a5305e"
format_stamp: "Formatted at 2026-09-29 11:01:44 on dist-test-slave-2lzq"
server_key: "f7b320c7754e8e63580725832f3b3fb3"
server_key_iv: "17f2f79eeaebb4ed7329cc26bc7bbd10"
server_key_version: "encryptionkey@0"
I20260929 11:01:45.061043 8877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:01:45.075423 8877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:01:45.077824 8877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:01:45.077994 8877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:01:45.082042 8877 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:43375
I20260929 11:01:45.082170 8945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:43375 every 8 connection(s)
I20260929 11:01:45.082564 8877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:01:45.084178 8946 sys_catalog.cc:263] Verifying existing consensus state
I20260929 11:01:45.084906 8946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e: Bootstrap starting.
I20260929 11:01:45.091257 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 8877
I20260929 11:01:45.091565 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.193:37655
--local_ip_for_outbound_sockets=127.26.120.193
--tserver_master_addrs=127.26.120.254:43375
--webserver_port=46413
--webserver_interface=127.26.120.193
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:33997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 11:01:45.093502 8946 log.cc:824] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e: Log is configured to *not* fsync() on all Append() calls
I20260929 11:01:45.095534 8946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 11:01:45.095861 8946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e: Bootstrap complete.
I20260929 11:01:45.098627 8946 raft_consensus.cc:348] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "099a625a94ff4147881504b477a5305e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43375 } }
I20260929 11:01:45.098887 8946 raft_consensus.cc:729] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 099a625a94ff4147881504b477a5305e, State: Initialized, Role: FOLLOWER
I20260929 11:01:45.099080 8946 consensus_queue.cc:260] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "099a625a94ff4147881504b477a5305e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43375 } }
I20260929 11:01:45.099160 8946 raft_consensus.cc:388] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 11:01:45.099228 8946 raft_consensus.cc:482] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 11:01:45.099303 8946 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [term 1 FOLLOWER]: Advancing to term 2
I20260929 11:01:45.100379 8946 raft_consensus.cc:504] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "099a625a94ff4147881504b477a5305e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43375 } }
I20260929 11:01:45.100507 8946 leader_election.cc:302] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 099a625a94ff4147881504b477a5305e; no voters:
I20260929 11:01:45.100720 8946 leader_election.cc:288] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 11:01:45.100757 8950 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [term 2 FOLLOWER]: Leader election won for term 2
I20260929 11:01:45.100872 8950 raft_consensus.cc:686] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [term 2 LEADER]: Becoming Leader. State: Replica: 099a625a94ff4147881504b477a5305e, State: Running, Role: LEADER
I20260929 11:01:45.100970 8950 consensus_queue.cc:237] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "099a625a94ff4147881504b477a5305e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43375 } }
I20260929 11:01:45.101006 8946 sys_catalog.cc:564] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:01:45.101228 8950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "099a625a94ff4147881504b477a5305e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "099a625a94ff4147881504b477a5305e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43375 } } }
I20260929 11:01:45.101294 8950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [sys.catalog]: This master's current role is: LEADER
I20260929 11:01:45.101408 8950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 099a625a94ff4147881504b477a5305e. Latest consensus state: current_term: 2 leader_uuid: "099a625a94ff4147881504b477a5305e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "099a625a94ff4147881504b477a5305e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43375 } } }
I20260929 11:01:45.101472 8950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [sys.catalog]: This master's current role is: LEADER
I20260929 11:01:45.101594 8954 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:01:45.102497 8954 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:01:45.102597 8954 catalog_manager.cc:1259] Loaded cluster ID: 95defd70f03a4de6b5c5db20a33104a1
I20260929 11:01:45.102624 8954 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 11:01:45.102844 8954 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260929 11:01:45.105808 8954 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260929 11:01:45.107765 8954 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:01:45.107995 8954 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e: Loaded TSK: 0
I20260929 11:01:45.108228 8954 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 11:01:45.189024 8948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:01:45.189257 8948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:01:45.189298 8948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:01:45.190528 8948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:01:45.190608 8948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:01:45.190661 8948 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.193
I20260929 11:01:45.192128 8948 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:33997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.193:37655
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.120.193
--webserver_port=46413
--tserver_master_addrs=127.26.120.254:43375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.193
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:01:45.192394 8948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:01:45.192627 8948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:01:45.198685 8974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:01:45.198706 8973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:01:45.198740 8948 server_base.cc:1034] running on GCE node
W20260929 11:01:45.198710 8976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:01:45.199178 8948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:01:45.199424 8948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:01:45.200569 8948 hybrid_clock.cc:648] HybridClock initialized: now 1790679705200552 us; error 32 us; skew 500 ppm
Sep 29 11:01:45 dist-test-slave-2lzq krb5kdc[8332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679705, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:01:45.203655 8948 init.cc:377] Logged in from keytab as kudu/127.26.120.193@KRBTEST.COM (short username kudu)
I20260929 11:01:45.204233 8948 webserver.cc:460] Webserver started at http://127.26.120.193:46413/ using document root <none> and password file <none>
I20260929 11:01:45.204459 8948 fs_manager.cc:359] Metadata directory not provided
I20260929 11:01:45.204545 8948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:01:45.205896 8948 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:01:45.206657 8983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:45.206840 8948 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 11:01:45.206966 8948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/wal
uuid: "3458dd5df95c4051b0962ec6aa134b41"
format_stamp: "Formatted at 2026-09-29 11:01:44 on dist-test-slave-2lzq"
server_key: "d1b56b76257e6d840e63b9488ef5a1d8"
server_key_iv: "a2d7e63b12db236689e4fdf07978d602"
server_key_version: "encryptionkey@0"
I20260929 11:01:45.207312 8948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:01:45.215976 8948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:01:45.218366 8948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:01:45.218520 8948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:01:45.218763 8948 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:01:45.219221 8990 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260929 11:01:45.221256 8948 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 11:01:45.221333 8948 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260929 11:01:45.221382 8948 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 11:01:45.222048 8948 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 11:01:45.222146 8948 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260929 11:01:45.222208 8990 tablet_bootstrap.cc:492] T 061863abbd6642719baf4c53c2311718 P 3458dd5df95c4051b0962ec6aa134b41: Bootstrap starting.
I20260929 11:01:45.228063 8948 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.193:37655
I20260929 11:01:45.228611 8948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
I20260929 11:01:45.229040 9097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.193:37655 every 8 connection(s)
I20260929 11:01:45.233245 8990 log.cc:824] T 061863abbd6642719baf4c53c2311718 P 3458dd5df95c4051b0962ec6aa134b41: Log is configured to *not* fsync() on all Append() calls
I20260929 11:01:45.235788 8990 tablet_bootstrap.cc:492] T 061863abbd6642719baf4c53c2311718 P 3458dd5df95c4051b0962ec6aa134b41: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 11:01:45.236172 8990 tablet_bootstrap.cc:492] T 061863abbd6642719baf4c53c2311718 P 3458dd5df95c4051b0962ec6aa134b41: Bootstrap complete.
Sep 29 11:01:45 dist-test-slave-2lzq krb5kdc[8332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679705, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:01:45.237071 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 8948
I20260929 11:01:45.237396 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.194:35157
--local_ip_for_outbound_sockets=127.26.120.194
--tserver_master_addrs=127.26.120.254:43375
--webserver_port=36835
--webserver_interface=127.26.120.194
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:33997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 11:01:45.237432 8990 ts_tablet_manager.cc:1397] T 061863abbd6642719baf4c53c2311718 P 3458dd5df95c4051b0962ec6aa134b41: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.011s
I20260929 11:01:45.238983 8990 raft_consensus.cc:348] T 061863abbd6642719baf4c53c2311718 P 3458dd5df95c4051b0962ec6aa134b41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d903e30c836441eab86034bebe7421d" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35157 } } peers { permanent_uuid: "3458dd5df95c4051b0962ec6aa134b41" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37655 } } peers { permanent_uuid: "d6b40d28ccab483f9cda4f705e51e737" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45339 } }
I20260929 11:01:45.239357 8990 raft_consensus.cc:729] T 061863abbd6642719baf4c53c2311718 P 3458dd5df95c4051b0962ec6aa134b41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3458dd5df95c4051b0962ec6aa134b41, State: Initialized, Role: FOLLOWER
I20260929 11:01:45.239539 8990 consensus_queue.cc:260] T 061863abbd6642719baf4c53c2311718 P 3458dd5df95c4051b0962ec6aa134b41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d903e30c836441eab86034bebe7421d" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35157 } } peers { permanent_uuid: "3458dd5df95c4051b0962ec6aa134b41" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37655 } } peers { permanent_uuid: "d6b40d28ccab483f9cda4f705e51e737" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45339 } }
I20260929 11:01:45.239882 8990 ts_tablet_manager.cc:1428] T 061863abbd6642719baf4c53c2311718 P 3458dd5df95c4051b0962ec6aa134b41: Time spent starting tablet: real 0.002s user 0.006s sys 0.000s
I20260929 11:01:45.241389 9100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:45.229714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.193:48837)
0929 11:01:45.229983 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:45.230018 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:45.230514 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:45.231010 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:45.231013 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:45.231295 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:01:45.231478 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:45.231483 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.232646 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.232647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:45.233526 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:45.233528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.234346 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.234347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:45.234507 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:45.235129 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:01:45.235148 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:01:45.237169 (+ 2021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:01:45.239639 (+ 2470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:45.239644 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:45.239646 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:45.239947 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:45.240226 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:45.240227 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:45.240227 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:45.240363 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:45.240756 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:01:45.240762 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:01:45.240893 (+ 131us) client_negotiation.cc:769] Sending connection context
0929 11:01:45.241013 (+ 120us) client_negotiation.cc:240] Negotiation successful
0929 11:01:45.241194 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":93,"threads_started":1}
I20260929 11:01:45.241413 9101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:45.229787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:48837 (local address 127.26.120.254:43375)
0929 11:01:45.230674 (+ 887us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:45.230681 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:45.230708 (+ 27us) server_negotiation.cc:407] Connection header received
0929 11:01:45.230805 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:45.230809 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:45.230846 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:45.230981 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:01:45.231606 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.232524 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.233624 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.234218 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.234333 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:45.237327 (+ 2994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:01:45.237331 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:01:45.237331 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:01:45.237344 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:01:45.239542 (+ 2198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:45.240044 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:45.240047 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:45.240047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:45.240145 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:45.240446 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:45.240447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:45.240448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:45.240677 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:01:45.240774 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:45.241065 (+ 291us) server_negotiation.cc:299] Negotiation successful
0929 11:01:45.241194 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":842,"thread_start_us":70,"threads_started":1}
I20260929 11:01:45.242322 9098 heartbeater.cc:347] Connected to a master server at 127.26.120.254:43375
I20260929 11:01:45.242419 9098 heartbeater.cc:464] Registering TS with master...
I20260929 11:01:45.242584 9098 heartbeater.cc:511] Master 127.26.120.254:43375 requested a full tablet report, sending...
I20260929 11:01:45.243424 8910 ts_manager.cc:194] Registered new tserver with Master: 3458dd5df95c4051b0962ec6aa134b41 (127.26.120.193:37655)
I20260929 11:01:45.244751 8910 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.193@KRBTEST.COM'} at 127.26.120.193:48837
I20260929 11:01:45.245855 9098 heartbeater.cc:503] Master 127.26.120.254:43375 was elected leader, sending a full tablet report...
I20260929 11:01:45.251317 9105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:45.244158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:37655 (local address 127.0.0.1:60200)
0929 11:01:45.244477 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:45.244510 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:45.244594 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:45.246306 (+ 1712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:45.246310 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:45.246312 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:01:45.246416 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:45.246418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.247706 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.247707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:45.249925 (+ 2218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:45.249929 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.250970 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.250971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:45.251084 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:45.251100 (+ 16us) client_negotiation.cc:769] Sending connection context
0929 11:01:45.251178 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 11:01:45.251233 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":144,"threads_started":1}
I20260929 11:01:45.251327 9106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:45.245137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60200 (local address 127.26.120.193:37655)
0929 11:01:45.246116 (+ 979us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:45.246120 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:45.246130 (+ 10us) server_negotiation.cc:407] Connection header received
0929 11:01:45.246168 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:45.246170 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:45.246186 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:45.246530 (+ 344us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:01:45.246573 (+ 43us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.247607 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.250043 (+ 2436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.250876 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.250962 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:45.251023 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:45.251205 (+ 182us) server_negotiation.cc:299] Negotiation successful
0929 11:01:45.251248 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":947,"thread_start_us":86,"threads_started":1}
I20260929 11:01:45.251925 9032 tablet_service.cc:1514] Processing DeleteTablet for tablet 061863abbd6642719baf4c53c2311718 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:01:44 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:60200
I20260929 11:01:45.252578 9107 tablet_replica.cc:330] stopping tablet replica
I20260929 11:01:45.252686 9107 raft_consensus.cc:2229] T 061863abbd6642719baf4c53c2311718 P 3458dd5df95c4051b0962ec6aa134b41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 11:01:45.252754 9107 raft_consensus.cc:2258] T 061863abbd6642719baf4c53c2311718 P 3458dd5df95c4051b0962ec6aa134b41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 11:01:45.253099 9107 ts_tablet_manager.cc:1905] T 061863abbd6642719baf4c53c2311718 P 3458dd5df95c4051b0962ec6aa134b41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 11:01:45.255409 9107 ts_tablet_manager.cc:1918] T 061863abbd6642719baf4c53c2311718 P 3458dd5df95c4051b0962ec6aa134b41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 11:01:45.255489 9107 log.cc:1196] T 061863abbd6642719baf4c53c2311718 P 3458dd5df95c4051b0962ec6aa134b41: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/wal/wals/061863abbd6642719baf4c53c2311718
I20260929 11:01:45.255718 9107 ts_tablet_manager.cc:1939] T 061863abbd6642719baf4c53c2311718 P 3458dd5df95c4051b0962ec6aa134b41: Deleting consensus metadata
I20260929 11:01:45.256280 8894 catalog_manager.cc:4841] TS 3458dd5df95c4051b0962ec6aa134b41 (127.26.120.193:37655): tablet 061863abbd6642719baf4c53c2311718 (table test-table [id=36299631cd8d4abf8ea1123bf612e76a]) successfully deleted
W20260929 11:01:45.323364 9103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:01:45.323606 9103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:01:45.323648 9103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:01:45.324878 9103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:01:45.324955 9103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:01:45.325001 9103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.194
I20260929 11:01:45.326483 9103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:33997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.194:35157
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.120.194
--webserver_port=36835
--tserver_master_addrs=127.26.120.254:43375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.194
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:01:45.326761 9103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:01:45.326990 9103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:01:45.332178 9115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:01:45.332333 9113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:01:45.332180 9112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:01:45.332432 9103 server_base.cc:1034] running on GCE node
I20260929 11:01:45.332618 9103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:01:45.332821 9103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:01:45.333990 9103 hybrid_clock.cc:648] HybridClock initialized: now 1790679705333947 us; error 36 us; skew 500 ppm
Sep 29 11:01:45 dist-test-slave-2lzq krb5kdc[8332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679705, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:01:45.336532 9103 init.cc:377] Logged in from keytab as kudu/127.26.120.194@KRBTEST.COM (short username kudu)
I20260929 11:01:45.336921 9103 webserver.cc:460] Webserver started at http://127.26.120.194:36835/ using document root <none> and password file <none>
I20260929 11:01:45.337111 9103 fs_manager.cc:359] Metadata directory not provided
I20260929 11:01:45.337163 9103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:01:45.338553 9103 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:01:45.339123 9122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:45.339308 9103 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 11:01:45.339416 9103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/wal
uuid: "3d903e30c836441eab86034bebe7421d"
format_stamp: "Formatted at 2026-09-29 11:01:44 on dist-test-slave-2lzq"
server_key: "dbf51f0d0dc37f38ec6b6f2742ae6e7b"
server_key_iv: "d169e87d10e0dd04f2f28eb3c05337ae"
server_key_version: "encryptionkey@0"
I20260929 11:01:45.339754 9103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:01:45.351544 9103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:01:45.353820 9103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:01:45.353983 9103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:01:45.354275 9103 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:01:45.354611 9103 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:01:45.354665 9103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:45.354708 9103 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:01:45.354735 9103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:45.360733 9103 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.194:35157
I20260929 11:01:45.360790 9235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.194:35157 every 8 connection(s)
I20260929 11:01:45.361145 9103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
I20260929 11:01:45.364506 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 9103
I20260929 11:01:45.364836 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.195:45339
--local_ip_for_outbound_sockets=127.26.120.195
--tserver_master_addrs=127.26.120.254:43375
--webserver_port=46053
--webserver_interface=127.26.120.195
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:33997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 11:01:45 dist-test-slave-2lzq krb5kdc[8332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679705, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:01:45.371657 9238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:45.361884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.194:59081)
0929 11:01:45.362024 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:45.362111 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:45.362659 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:45.362919 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:45.362923 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:45.363493 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:01:45.363766 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:45.363771 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.364536 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.364537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:45.365597 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:45.365599 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.366080 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.366082 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:45.366246 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:45.366889 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:01:45.366898 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:01:45.368288 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:01:45.370204 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:45.370216 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:45.370218 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:45.370442 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:45.370661 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:45.370662 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:45.370662 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:45.370764 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:45.371074 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:01:45.371078 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:01:45.371271 (+ 193us) client_negotiation.cc:769] Sending connection context
0929 11:01:45.371361 (+ 90us) client_negotiation.cc:240] Negotiation successful
0929 11:01:45.371493 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":69,"threads_started":1}
I20260929 11:01:45.371665 9101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:45.361913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:59081 (local address 127.26.120.254:43375)
0929 11:01:45.361957 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:45.361959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:45.362206 (+ 247us) server_negotiation.cc:407] Connection header received
0929 11:01:45.362769 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:45.362771 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:45.362785 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:45.362839 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:01:45.363880 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.364441 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.365688 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.365951 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.366024 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:45.368390 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:01:45.368393 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:01:45.368393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:01:45.368401 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:01:45.370122 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:45.370521 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:45.370523 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:45.370523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:45.370590 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:45.370834 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:45.370835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:45.370836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:45.371003 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:01:45.371082 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:45.371420 (+ 338us) server_negotiation.cc:299] Negotiation successful
0929 11:01:45.371525 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 11:01:45.372153 9236 heartbeater.cc:347] Connected to a master server at 127.26.120.254:43375
I20260929 11:01:45.372267 9236 heartbeater.cc:464] Registering TS with master...
I20260929 11:01:45.372488 9236 heartbeater.cc:511] Master 127.26.120.254:43375 requested a full tablet report, sending...
I20260929 11:01:45.372884 8910 ts_manager.cc:194] Registered new tserver with Master: 3d903e30c836441eab86034bebe7421d (127.26.120.194:35157)
I20260929 11:01:45.373257 8910 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.194@KRBTEST.COM'} at 127.26.120.194:59081
W20260929 11:01:45.447345 9239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:01:45.447543 9239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:01:45.447566 9239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:01:45.448694 9239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:01:45.448735 9239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:01:45.448762 9239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.195
I20260929 11:01:45.450246 9239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:33997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.195:45339
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.120.195
--webserver_port=46053
--tserver_master_addrs=127.26.120.254:43375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.195
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:01:45.450572 9239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:01:45.450826 9239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:01:45.455804 9245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:01:45.455891 9244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:01:45.455953 9239 server_base.cc:1034] running on GCE node
W20260929 11:01:45.455899 9247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:01:45.456244 9239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:01:45.456504 9239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:01:45.457679 9239 hybrid_clock.cc:648] HybridClock initialized: now 1790679705457664 us; error 29 us; skew 500 ppm
Sep 29 11:01:45 dist-test-slave-2lzq krb5kdc[8332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679705, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:01:45.460357 9239 init.cc:377] Logged in from keytab as kudu/127.26.120.195@KRBTEST.COM (short username kudu)
I20260929 11:01:45.460752 9239 webserver.cc:460] Webserver started at http://127.26.120.195:46053/ using document root <none> and password file <none>
I20260929 11:01:45.460942 9239 fs_manager.cc:359] Metadata directory not provided
I20260929 11:01:45.460994 9239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:01:45.462713 9239 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:01:45.463261 9254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:45.463449 9239 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:45.463521 9239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/wal
uuid: "d6b40d28ccab483f9cda4f705e51e737"
format_stamp: "Formatted at 2026-09-29 11:01:44 on dist-test-slave-2lzq"
server_key: "6c622702ddcc679d35bd0e0ac86ab89c"
server_key_iv: "0d84a0d2a688f8ef0207922b61ce0b08"
server_key_version: "encryptionkey@0"
I20260929 11:01:45.463944 9239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:01:45.487133 9239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:01:45.489682 9239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:01:45.489867 9239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:01:45.490206 9239 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:01:45.490540 9239 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:01:45.490597 9239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:45.490638 9239 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:01:45.490664 9239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:45.496583 9239 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.195:45339
I20260929 11:01:45.496660 9367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.195:45339 every 8 connection(s)
I20260929 11:01:45.497037 9239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
I20260929 11:01:45.499944 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 9239
Sep 29 11:01:45 dist-test-slave-2lzq krb5kdc[8332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679705, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:01:45.507283 9372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:45.500365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39016 (local address 127.26.120.254:43375)
0929 11:01:45.500525 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:45.500527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:45.500537 (+ 10us) server_negotiation.cc:407] Connection header received
0929 11:01:45.500588 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:45.500589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:45.500599 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:45.500664 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:01:45.501306 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.502080 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.502803 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.503016 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.503090 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:45.504426 (+ 1336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:01:45.504427 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:01:45.504428 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:01:45.504435 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:01:45.506028 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:45.506413 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:45.506414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:45.506415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:45.506496 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:45.506716 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:45.506717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:45.506717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:45.506891 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:01:45.507062 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:45.507099 (+ 37us) server_negotiation.cc:299] Negotiation successful
0929 11:01:45.507183 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":74,"threads_started":1}
I20260929 11:01:45.507440 9101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:45.497837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:50723 (local address 127.26.120.254:43375)
0929 11:01:45.497917 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:45.497918 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:45.498178 (+ 260us) server_negotiation.cc:407] Connection header received
0929 11:01:45.498665 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:45.498666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:45.498677 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:45.498741 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:01:45.499703 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.500422 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.501562 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.501856 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.501930 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:45.504205 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:01:45.504206 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:01:45.504207 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:01:45.504212 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:01:45.505855 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:45.506328 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:45.506329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:45.506330 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:45.506387 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:45.506706 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:45.506707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:45.506708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:45.506832 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:01:45.506880 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:45.507235 (+ 355us) server_negotiation.cc:299] Negotiation successful
0929 11:01:45.507311 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 11:01:45.507592 9370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:45.497827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.195:50723)
0929 11:01:45.498012 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:45.498085 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:45.498581 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:45.498810 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:45.498813 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:45.499420 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:01:45.499618 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:45.499623 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.500511 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.500512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:45.501447 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:45.501450 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.501945 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.501945 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:45.502114 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:45.502685 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:01:45.502694 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:01:45.504095 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:01:45.505936 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:45.505943 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:45.505946 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:45.506258 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:45.506456 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:45.506457 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:45.506457 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:45.506607 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:45.506901 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:01:45.506907 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:01:45.507065 (+ 158us) client_negotiation.cc:769] Sending connection context
0929 11:01:45.507175 (+ 110us) client_negotiation.cc:240] Negotiation successful
0929 11:01:45.507336 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":111,"threads_started":1}
I20260929 11:01:45.508034 9368 heartbeater.cc:347] Connected to a master server at 127.26.120.254:43375
I20260929 11:01:45.508143 9368 heartbeater.cc:464] Registering TS with master...
I20260929 11:01:45.508329 9368 heartbeater.cc:511] Master 127.26.120.254:43375 requested a full tablet report, sending...
I20260929 11:01:45.508669 8910 ts_manager.cc:194] Registered new tserver with Master: d6b40d28ccab483f9cda4f705e51e737 (127.26.120.195:45339)
I20260929 11:01:45.508998 8910 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.195@KRBTEST.COM'} at 127.26.120.195:50723
I20260929 11:01:45.519201 27107 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 11:01:45.522006 9101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:45.519602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39030 (local address 127.26.120.254:43375)
0929 11:01:45.519667 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:45.519668 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:45.519787 (+ 119us) server_negotiation.cc:407] Connection header received
0929 11:01:45.519876 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:45.519876 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:45.519911 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:45.519979 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 11:01:45.520084 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.520554 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.521366 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.521546 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.521595 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:45.521742 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 11:01:45.521839 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 11:01:45.521899 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:45.521935 (+ 36us) server_negotiation.cc:299] Negotiation successful
0929 11:01:45.521950 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260929 11:01:45.522327 8910 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:39030:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 11:01:45.523924 8910 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 11:01:45.526042 9032 tablet_service.cc:1467] Processing CreateTablet for tablet cfa300a616c5446e820e1732db151f88 (DEFAULT_TABLE table=test-table [id=f0ab802e55a84faabd37472948914018]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:01:45.526268 9032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfa300a616c5446e820e1732db151f88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:01:45.527724 8990 tablet_bootstrap.cc:492] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41: Bootstrap starting.
I20260929 11:01:45.528035 8990 tablet_bootstrap.cc:654] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41: Neither blocks nor log segments found. Creating new log.
I20260929 11:01:45.528616 8990 tablet_bootstrap.cc:492] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41: No bootstrap required, opened a new log
I20260929 11:01:45.528688 8990 ts_tablet_manager.cc:1397] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 11:01:45.528817 8990 raft_consensus.cc:348] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6b40d28ccab483f9cda4f705e51e737" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45339 } } peers { permanent_uuid: "3d903e30c836441eab86034bebe7421d" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35157 } } peers { permanent_uuid: "3458dd5df95c4051b0962ec6aa134b41" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37655 } }
I20260929 11:01:45.528903 8990 raft_consensus.cc:374] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:01:45.528950 8990 raft_consensus.cc:729] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3458dd5df95c4051b0962ec6aa134b41, State: Initialized, Role: FOLLOWER
I20260929 11:01:45.528995 8990 consensus_queue.cc:260] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6b40d28ccab483f9cda4f705e51e737" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45339 } } peers { permanent_uuid: "3d903e30c836441eab86034bebe7421d" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35157 } } peers { permanent_uuid: "3458dd5df95c4051b0962ec6aa134b41" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37655 } }
I20260929 11:01:45.529086 8990 ts_tablet_manager.cc:1428] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260929 11:01:45.530308 9376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:45.526021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39352 (local address 127.26.120.194:35157)
0929 11:01:45.526233 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:45.526237 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:45.526246 (+ 9us) server_negotiation.cc:407] Connection header received
0929 11:01:45.526280 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:45.526282 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:45.526300 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:45.526372 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:01:45.526586 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.527694 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.529183 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.529925 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.530007 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:45.530097 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:45.530183 (+ 86us) server_negotiation.cc:299] Negotiation successful
0929 11:01:45.530222 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":105,"threads_started":1}
I20260929 11:01:45.530308 9375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:45.525921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45120 (local address 127.26.120.195:45339)
0929 11:01:45.526189 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:45.526195 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:45.526206 (+ 11us) server_negotiation.cc:407] Connection header received
0929 11:01:45.526251 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:45.526253 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:45.526271 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:45.526516 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:01:45.526599 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.527693 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.529183 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.529927 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.530007 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:45.530088 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:45.530183 (+ 95us) server_negotiation.cc:299] Negotiation successful
0929 11:01:45.530222 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":125,"threads_started":1}
I20260929 11:01:45.530309 9105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:45.525739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:45339 (local address 127.0.0.1:45120)
0929 11:01:45.525785 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:45.525798 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:45.525834 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:45.526400 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:45.526401 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:45.526403 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:01:45.526444 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:45.526445 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.527816 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.527816 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:45.529084 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:45.529086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.530018 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.530018 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:45.530119 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:45.530122 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 11:01:45.530171 (+ 49us) client_negotiation.cc:240] Negotiation successful
0929 11:01:45.530221 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":61}
I20260929 11:01:45.530310 9374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:45.525766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:35157 (local address 127.0.0.1:39352)
0929 11:01:45.526027 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:45.526038 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:45.526117 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:45.526436 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:45.526437 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:45.526439 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:01:45.526487 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:45.526488 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.527813 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.527825 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:45.529084 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:45.529086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.530013 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.530014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:45.530105 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:45.530110 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 11:01:45.530171 (+ 61us) client_negotiation.cc:240] Negotiation successful
0929 11:01:45.530221 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":67,"thread_start_us":95,"threads_started":1}
I20260929 11:01:45.531025 9170 tablet_service.cc:1467] Processing CreateTablet for tablet cfa300a616c5446e820e1732db151f88 (DEFAULT_TABLE table=test-table [id=f0ab802e55a84faabd37472948914018]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:01:45.531181 9302 tablet_service.cc:1467] Processing CreateTablet for tablet cfa300a616c5446e820e1732db151f88 (DEFAULT_TABLE table=test-table [id=f0ab802e55a84faabd37472948914018]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:01:45.531363 9170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfa300a616c5446e820e1732db151f88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:01:45.531538 9302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfa300a616c5446e820e1732db151f88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:01:45.534013 9378 tablet_bootstrap.cc:492] T cfa300a616c5446e820e1732db151f88 P 3d903e30c836441eab86034bebe7421d: Bootstrap starting.
I20260929 11:01:45.534021 9379 tablet_bootstrap.cc:492] T cfa300a616c5446e820e1732db151f88 P d6b40d28ccab483f9cda4f705e51e737: Bootstrap starting.
I20260929 11:01:45.534709 9378 tablet_bootstrap.cc:654] T cfa300a616c5446e820e1732db151f88 P 3d903e30c836441eab86034bebe7421d: Neither blocks nor log segments found. Creating new log.
I20260929 11:01:45.534780 9379 tablet_bootstrap.cc:654] T cfa300a616c5446e820e1732db151f88 P d6b40d28ccab483f9cda4f705e51e737: Neither blocks nor log segments found. Creating new log.
I20260929 11:01:45.535239 9378 log.cc:824] T cfa300a616c5446e820e1732db151f88 P 3d903e30c836441eab86034bebe7421d: Log is configured to *not* fsync() on all Append() calls
I20260929 11:01:45.535260 9379 log.cc:824] T cfa300a616c5446e820e1732db151f88 P d6b40d28ccab483f9cda4f705e51e737: Log is configured to *not* fsync() on all Append() calls
I20260929 11:01:45.535948 9379 tablet_bootstrap.cc:492] T cfa300a616c5446e820e1732db151f88 P d6b40d28ccab483f9cda4f705e51e737: No bootstrap required, opened a new log
I20260929 11:01:45.536057 9379 ts_tablet_manager.cc:1397] T cfa300a616c5446e820e1732db151f88 P d6b40d28ccab483f9cda4f705e51e737: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 11:01:45.536157 9378 tablet_bootstrap.cc:492] T cfa300a616c5446e820e1732db151f88 P 3d903e30c836441eab86034bebe7421d: No bootstrap required, opened a new log
I20260929 11:01:45.536245 9378 ts_tablet_manager.cc:1397] T cfa300a616c5446e820e1732db151f88 P 3d903e30c836441eab86034bebe7421d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 11:01:45.537570 9379 raft_consensus.cc:348] T cfa300a616c5446e820e1732db151f88 P d6b40d28ccab483f9cda4f705e51e737 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6b40d28ccab483f9cda4f705e51e737" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45339 } } peers { permanent_uuid: "3d903e30c836441eab86034bebe7421d" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35157 } } peers { permanent_uuid: "3458dd5df95c4051b0962ec6aa134b41" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37655 } }
I20260929 11:01:45.537631 9378 raft_consensus.cc:348] T cfa300a616c5446e820e1732db151f88 P 3d903e30c836441eab86034bebe7421d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6b40d28ccab483f9cda4f705e51e737" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45339 } } peers { permanent_uuid: "3d903e30c836441eab86034bebe7421d" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35157 } } peers { permanent_uuid: "3458dd5df95c4051b0962ec6aa134b41" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37655 } }
I20260929 11:01:45.537760 9379 raft_consensus.cc:374] T cfa300a616c5446e820e1732db151f88 P d6b40d28ccab483f9cda4f705e51e737 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:01:45.537797 9378 raft_consensus.cc:374] T cfa300a616c5446e820e1732db151f88 P 3d903e30c836441eab86034bebe7421d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:01:45.537803 9379 raft_consensus.cc:729] T cfa300a616c5446e820e1732db151f88 P d6b40d28ccab483f9cda4f705e51e737 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6b40d28ccab483f9cda4f705e51e737, State: Initialized, Role: FOLLOWER
I20260929 11:01:45.537861 9378 raft_consensus.cc:729] T cfa300a616c5446e820e1732db151f88 P 3d903e30c836441eab86034bebe7421d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d903e30c836441eab86034bebe7421d, State: Initialized, Role: FOLLOWER
I20260929 11:01:45.537981 9379 consensus_queue.cc:260] T cfa300a616c5446e820e1732db151f88 P d6b40d28ccab483f9cda4f705e51e737 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6b40d28ccab483f9cda4f705e51e737" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45339 } } peers { permanent_uuid: "3d903e30c836441eab86034bebe7421d" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35157 } } peers { permanent_uuid: "3458dd5df95c4051b0962ec6aa134b41" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37655 } }
I20260929 11:01:45.538004 9378 consensus_queue.cc:260] T cfa300a616c5446e820e1732db151f88 P 3d903e30c836441eab86034bebe7421d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6b40d28ccab483f9cda4f705e51e737" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45339 } } peers { permanent_uuid: "3d903e30c836441eab86034bebe7421d" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35157 } } peers { permanent_uuid: "3458dd5df95c4051b0962ec6aa134b41" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37655 } }
I20260929 11:01:45.538329 9378 ts_tablet_manager.cc:1428] T cfa300a616c5446e820e1732db151f88 P 3d903e30c836441eab86034bebe7421d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 11:01:45.538340 9379 ts_tablet_manager.cc:1428] T cfa300a616c5446e820e1732db151f88 P d6b40d28ccab483f9cda4f705e51e737: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 11:01:45.538367 9236 heartbeater.cc:503] Master 127.26.120.254:43375 was elected leader, sending a full tablet report...
I20260929 11:01:45.538437 9368 heartbeater.cc:503] Master 127.26.120.254:43375 was elected leader, sending a full tablet report...
W20260929 11:01:45.611707 9237 tablet.cc:2369] T cfa300a616c5446e820e1732db151f88 P 3d903e30c836441eab86034bebe7421d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 11:01:45.694718 9382 raft_consensus.cc:482] T cfa300a616c5446e820e1732db151f88 P 3d903e30c836441eab86034bebe7421d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 11:01:45.694844 9382 raft_consensus.cc:504] T cfa300a616c5446e820e1732db151f88 P 3d903e30c836441eab86034bebe7421d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6b40d28ccab483f9cda4f705e51e737" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45339 } } peers { permanent_uuid: "3d903e30c836441eab86034bebe7421d" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35157 } } peers { permanent_uuid: "3458dd5df95c4051b0962ec6aa134b41" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37655 } }
I20260929 11:01:45.695207 9382 leader_election.cc:288] T cfa300a616c5446e820e1732db151f88 P 3d903e30c836441eab86034bebe7421d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6b40d28ccab483f9cda4f705e51e737 (127.26.120.195:45339), 3458dd5df95c4051b0962ec6aa134b41 (127.26.120.193:37655)
I20260929 11:01:45.699261 9375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:45.695516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:33531 (local address 127.26.120.195:45339)
0929 11:01:45.695562 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:45.695564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:45.695578 (+ 14us) server_negotiation.cc:407] Connection header received
0929 11:01:45.695618 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:45.695619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:45.695627 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:45.695712 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:01:45.695922 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.696680 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.698330 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.698909 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.698983 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:45.698992 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:45.699150 (+ 158us) server_negotiation.cc:299] Negotiation successful
0929 11:01:45.699189 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 11:01:45.699261 9106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:45.695501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:45697 (local address 127.26.120.193:37655)
0929 11:01:45.695570 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:45.695572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:45.695670 (+ 98us) server_negotiation.cc:407] Connection header received
0929 11:01:45.695701 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:45.695702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:45.695712 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:45.695767 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:01:45.695922 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.696698 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.698331 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.698902 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.698983 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:45.698992 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:45.699150 (+ 158us) server_negotiation.cc:299] Negotiation successful
0929 11:01:45.699193 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 11:01:45.699291 9238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:45.695404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:45339 (local address 127.26.120.194:33531)
0929 11:01:45.695468 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:45.695483 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:45.695554 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:45.695732 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:45.695732 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:45.695734 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:01:45.695831 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:45.695833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.696772 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.696773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:45.698230 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:45.698234 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.698992 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.698993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:45.699075 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:45.699076 (+ 1us) client_negotiation.cc:769] Sending connection context
0929 11:01:45.699133 (+ 57us) client_negotiation.cc:240] Negotiation successful
0929 11:01:45.699181 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":80}
I20260929 11:01:45.699297 9384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:45.695421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:37655 (local address 127.26.120.194:45697)
0929 11:01:45.695594 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:45.695604 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:45.695658 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:45.695778 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:45.695779 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:45.695780 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:01:45.695837 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:45.695838 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.696772 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.696773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:45.698230 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:45.698234 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.698992 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.698993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:45.699071 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:45.699076 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 11:01:45.699133 (+ 57us) client_negotiation.cc:240] Negotiation successful
0929 11:01:45.699180 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":92,"thread_start_us":103,"threads_started":1}
I20260929 11:01:45.699666 9322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa300a616c5446e820e1732db151f88" candidate_uuid: "3d903e30c836441eab86034bebe7421d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6b40d28ccab483f9cda4f705e51e737" is_pre_election: true
I20260929 11:01:45.699849 9322 raft_consensus.cc:2454] T cfa300a616c5446e820e1732db151f88 P d6b40d28ccab483f9cda4f705e51e737 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d903e30c836441eab86034bebe7421d in term 0.
I20260929 11:01:45.700032 9052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa300a616c5446e820e1732db151f88" candidate_uuid: "3d903e30c836441eab86034bebe7421d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3458dd5df95c4051b0962ec6aa134b41" is_pre_election: true
I20260929 11:01:45.700106 9125 leader_election.cc:302] T cfa300a616c5446e820e1732db151f88 P 3d903e30c836441eab86034bebe7421d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d903e30c836441eab86034bebe7421d, d6b40d28ccab483f9cda4f705e51e737; no voters:
I20260929 11:01:45.700212 9052 raft_consensus.cc:2454] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d903e30c836441eab86034bebe7421d in term 0.
I20260929 11:01:45.700245 9382 raft_consensus.cc:2781] T cfa300a616c5446e820e1732db151f88 P 3d903e30c836441eab86034bebe7421d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 11:01:45.700297 9382 raft_consensus.cc:482] T cfa300a616c5446e820e1732db151f88 P 3d903e30c836441eab86034bebe7421d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 11:01:45.700327 9382 raft_consensus.cc:3037] T cfa300a616c5446e820e1732db151f88 P 3d903e30c836441eab86034bebe7421d [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:01:45.701406 9382 raft_consensus.cc:504] T cfa300a616c5446e820e1732db151f88 P 3d903e30c836441eab86034bebe7421d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6b40d28ccab483f9cda4f705e51e737" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45339 } } peers { permanent_uuid: "3d903e30c836441eab86034bebe7421d" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35157 } } peers { permanent_uuid: "3458dd5df95c4051b0962ec6aa134b41" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37655 } }
I20260929 11:01:45.701598 9382 leader_election.cc:288] T cfa300a616c5446e820e1732db151f88 P 3d903e30c836441eab86034bebe7421d [CANDIDATE]: Term 1 election: Requested vote from peers d6b40d28ccab483f9cda4f705e51e737 (127.26.120.195:45339), 3458dd5df95c4051b0962ec6aa134b41 (127.26.120.193:37655)
I20260929 11:01:45.701745 9322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa300a616c5446e820e1732db151f88" candidate_uuid: "3d903e30c836441eab86034bebe7421d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6b40d28ccab483f9cda4f705e51e737"
I20260929 11:01:45.701766 9052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa300a616c5446e820e1732db151f88" candidate_uuid: "3d903e30c836441eab86034bebe7421d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3458dd5df95c4051b0962ec6aa134b41"
I20260929 11:01:45.701835 9052 raft_consensus.cc:3037] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:01:45.701830 9322 raft_consensus.cc:3037] T cfa300a616c5446e820e1732db151f88 P d6b40d28ccab483f9cda4f705e51e737 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:01:45.702745 9322 raft_consensus.cc:2454] T cfa300a616c5446e820e1732db151f88 P d6b40d28ccab483f9cda4f705e51e737 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d903e30c836441eab86034bebe7421d in term 1.
I20260929 11:01:45.702745 9052 raft_consensus.cc:2454] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d903e30c836441eab86034bebe7421d in term 1.
I20260929 11:01:45.702926 9123 leader_election.cc:302] T cfa300a616c5446e820e1732db151f88 P 3d903e30c836441eab86034bebe7421d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3458dd5df95c4051b0962ec6aa134b41, 3d903e30c836441eab86034bebe7421d; no voters:
I20260929 11:01:45.703040 9382 raft_consensus.cc:2781] T cfa300a616c5446e820e1732db151f88 P 3d903e30c836441eab86034bebe7421d [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:01:45.703208 9382 raft_consensus.cc:686] T cfa300a616c5446e820e1732db151f88 P 3d903e30c836441eab86034bebe7421d [term 1 LEADER]: Becoming Leader. State: Replica: 3d903e30c836441eab86034bebe7421d, State: Running, Role: LEADER
I20260929 11:01:45.703320 9382 consensus_queue.cc:237] T cfa300a616c5446e820e1732db151f88 P 3d903e30c836441eab86034bebe7421d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6b40d28ccab483f9cda4f705e51e737" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45339 } } peers { permanent_uuid: "3d903e30c836441eab86034bebe7421d" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35157 } } peers { permanent_uuid: "3458dd5df95c4051b0962ec6aa134b41" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37655 } }
I20260929 11:01:45.704186 8910 catalog_manager.cc:5495] T cfa300a616c5446e820e1732db151f88 P 3d903e30c836441eab86034bebe7421d reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d903e30c836441eab86034bebe7421d (127.26.120.194). New cstate: current_term: 1 leader_uuid: "3d903e30c836441eab86034bebe7421d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6b40d28ccab483f9cda4f705e51e737" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d903e30c836441eab86034bebe7421d" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3458dd5df95c4051b0962ec6aa134b41" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37655 } health_report { overall_health: UNKNOWN } } }
I20260929 11:01:45.710234 9376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:45.707962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39354 (local address 127.26.120.194:35157)
0929 11:01:45.708001 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:45.708003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:45.708011 (+ 8us) server_negotiation.cc:407] Connection header received
0929 11:01:45.708079 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:45.708080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:45.708096 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:45.708152 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 11:01:45.708247 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.708694 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.709385 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:45.709643 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:45.709696 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:45.709820 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 11:01:45.709912 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 11:01:45.710028 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:45.710043 (+ 15us) server_negotiation.cc:299] Negotiation successful
0929 11:01:45.710105 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 11:01:45.711963 9052 raft_consensus.cc:1261] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [term 1 FOLLOWER]: Refusing update from remote peer 3d903e30c836441eab86034bebe7421d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 11:01:45.711982 9322 raft_consensus.cc:1261] T cfa300a616c5446e820e1732db151f88 P d6b40d28ccab483f9cda4f705e51e737 [term 1 FOLLOWER]: Refusing update from remote peer 3d903e30c836441eab86034bebe7421d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 11:01:45.712296 9382 consensus_queue.cc:1035] T cfa300a616c5446e820e1732db151f88 P 3d903e30c836441eab86034bebe7421d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6b40d28ccab483f9cda4f705e51e737" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 11:01:45.712436 9385 consensus_queue.cc:1035] T cfa300a616c5446e820e1732db151f88 P 3d903e30c836441eab86034bebe7421d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3458dd5df95c4051b0962ec6aa134b41" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 11:01:45.714603 9387 mvcc.cc:205] Tried to move back new op lower bound from 7334624074593632256 to 7334624074561552384. Current Snapshot: MvccSnapshot[applied={T|T < 7334624074593632256}]
I20260929 11:01:45.717038 8910 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:39030:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 11:01:45.717125 8910 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:39030:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 11:01:45.717820 8910 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e: Sending DeleteTablet for 3 replicas of tablet cfa300a616c5446e820e1732db151f88
I20260929 11:01:45.718134 9032 tablet_service.cc:1514] Processing DeleteTablet for tablet cfa300a616c5446e820e1732db151f88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:01:45 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:60200
I20260929 11:01:45.718139 9302 tablet_service.cc:1514] Processing DeleteTablet for tablet cfa300a616c5446e820e1732db151f88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:01:45 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:45120
I20260929 11:01:45.718287 9107 tablet_replica.cc:330] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41: stopping tablet replica
I20260929 11:01:45.718303 9170 tablet_service.cc:1514] Processing DeleteTablet for tablet cfa300a616c5446e820e1732db151f88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:01:45 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:39352
I20260929 11:01:45.718350 9107 raft_consensus.cc:2229] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 11:01:45.718394 9107 raft_consensus.cc:2258] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 11:01:45.718506 9397 tablet_replica.cc:330] T cfa300a616c5446e820e1732db151f88 P 3d903e30c836441eab86034bebe7421d: stopping tablet replica
I20260929 11:01:45.718551 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 8948
I20260929 11:01:45.718590 9397 raft_consensus.cc:2229] T cfa300a616c5446e820e1732db151f88 P 3d903e30c836441eab86034bebe7421d [term 1 LEADER]: Raft consensus shutting down.
I20260929 11:01:45.719054 9397 raft_consensus.cc:2258] T cfa300a616c5446e820e1732db151f88 P 3d903e30c836441eab86034bebe7421d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 11:01:45.719489 9397 ts_tablet_manager.cc:1905] T cfa300a616c5446e820e1732db151f88 P 3d903e30c836441eab86034bebe7421d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 11:01:45.721792 9397 ts_tablet_manager.cc:1918] T cfa300a616c5446e820e1732db151f88 P 3d903e30c836441eab86034bebe7421d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 11:01:45.721872 9397 log.cc:1196] T cfa300a616c5446e820e1732db151f88 P 3d903e30c836441eab86034bebe7421d: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/wal/wals/cfa300a616c5446e820e1732db151f88
I20260929 11:01:45.722199 9397 ts_tablet_manager.cc:1939] T cfa300a616c5446e820e1732db151f88 P 3d903e30c836441eab86034bebe7421d: Deleting consensus metadata
I20260929 11:01:45.722622 8895 catalog_manager.cc:4841] TS 3d903e30c836441eab86034bebe7421d (127.26.120.194:35157): tablet cfa300a616c5446e820e1732db151f88 (table test-table [id=f0ab802e55a84faabd37472948914018]) successfully deleted
W20260929 11:01:45.725570 8894 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.120.193:37655 (error 108)
I20260929 11:01:45.725732 9396 tablet_replica.cc:330] T cfa300a616c5446e820e1732db151f88 P d6b40d28ccab483f9cda4f705e51e737: stopping tablet replica
I20260929 11:01:45.725873 9396 raft_consensus.cc:2229] T cfa300a616c5446e820e1732db151f88 P d6b40d28ccab483f9cda4f705e51e737 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 11:01:45.726017 9396 raft_consensus.cc:2258] T cfa300a616c5446e820e1732db151f88 P d6b40d28ccab483f9cda4f705e51e737 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 11:01:45.726173 9374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:45.726024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:37655 (local address 127.0.0.1:37092)
0929 11:01:45.726094 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:45.726137 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.193:37655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260929 11:01:45.726261 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 9103
W20260929 11:01:45.726291 8894 catalog_manager.cc:4584] TS 3458dd5df95c4051b0962ec6aa134b41 (127.26.120.193:37655): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cfa300a616c5446e820e1732db151f88: Network error: Client connection negotiation failed: client connection to 127.26.120.193:37655: connect: Connection refused (error 111)
I20260929 11:01:45.730465 9396 ts_tablet_manager.cc:1905] T cfa300a616c5446e820e1732db151f88 P d6b40d28ccab483f9cda4f705e51e737: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 11:01:45.731667 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 9239
I20260929 11:01:45.736166 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 8877
I20260929 11:01:45.736266 9374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:45.736126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:45339 (local address 127.0.0.1:45136)
0929 11:01:45.736184 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:45.736217 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.195:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260929 11:01:45.741446 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:43375
--webserver_interface=127.26.120.254
--webserver_port=37609
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:33997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:43375
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:01:45.826524 9399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:01:45.826745 9399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:01:45.826777 9399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:01:45.827986 9399 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:01:45.828037 9399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:01:45.828052 9399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:01:45.828066 9399 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:01:45.828078 9399 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:01:45.829612 9399 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:33997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:43375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:43375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=37609
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:01:45.829872 9399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:01:45.830161 9399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:01:45.835219 9405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:01:45.835213 9407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:01:45.835210 9404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:01:45.835551 9399 server_base.cc:1034] running on GCE node
I20260929 11:01:45.835806 9399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:01:45.836117 9399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:01:45.837291 9399 hybrid_clock.cc:648] HybridClock initialized: now 1790679705837268 us; error 39 us; skew 500 ppm
Sep 29 11:01:45 dist-test-slave-2lzq krb5kdc[8332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679705, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:01:45.839819 9399 init.cc:377] Logged in from keytab as kudu/127.26.120.254@KRBTEST.COM (short username kudu)
I20260929 11:01:45.840294 9399 webserver.cc:460] Webserver started at http://127.26.120.254:37609/ using document root <none> and password file <none>
I20260929 11:01:45.840546 9399 fs_manager.cc:359] Metadata directory not provided
I20260929 11:01:45.840617 9399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:01:45.841984 9399 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:01:45.842749 9414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:45.842939 9399 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:01:45.843043 9399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "099a625a94ff4147881504b477a5305e"
format_stamp: "Formatted at 2026-09-29 11:01:44 on dist-test-slave-2lzq"
server_key: "f7b320c7754e8e63580725832f3b3fb3"
server_key_iv: "17f2f79eeaebb4ed7329cc26bc7bbd10"
server_key_version: "encryptionkey@0"
I20260929 11:01:45.843415 9399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:01:45.860997 9399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:01:45.862982 9399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:01:45.863144 9399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:01:45.867378 9399 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:43375
I20260929 11:01:45.867414 9466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:43375 every 8 connection(s)
I20260929 11:01:45.867797 9399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:01:45.869330 9467 sys_catalog.cc:263] Verifying existing consensus state
I20260929 11:01:45.869910 9467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e: Bootstrap starting.
I20260929 11:01:45.876456 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 9399
I20260929 11:01:45.876799 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.193:37655
--local_ip_for_outbound_sockets=127.26.120.193
--tserver_master_addrs=127.26.120.254:43375
--webserver_port=46413
--webserver_interface=127.26.120.193
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:33997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 11:01:45.878898 9467 log.cc:824] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e: Log is configured to *not* fsync() on all Append() calls
I20260929 11:01:45.881258 9467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 11:01:45.881564 9467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e: Bootstrap complete.
I20260929 11:01:45.883839 9467 raft_consensus.cc:348] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "099a625a94ff4147881504b477a5305e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43375 } }
I20260929 11:01:45.884051 9467 raft_consensus.cc:729] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 099a625a94ff4147881504b477a5305e, State: Initialized, Role: FOLLOWER
I20260929 11:01:45.884219 9467 consensus_queue.cc:260] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "099a625a94ff4147881504b477a5305e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43375 } }
I20260929 11:01:45.884279 9467 raft_consensus.cc:388] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 11:01:45.884341 9467 raft_consensus.cc:482] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 11:01:45.884399 9467 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [term 2 FOLLOWER]: Advancing to term 3
I20260929 11:01:45.885375 9467 raft_consensus.cc:504] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "099a625a94ff4147881504b477a5305e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43375 } }
I20260929 11:01:45.885456 9467 leader_election.cc:302] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 099a625a94ff4147881504b477a5305e; no voters:
I20260929 11:01:45.885630 9467 leader_election.cc:288] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [CANDIDATE]: Term 3 election: Requested vote from peers
I20260929 11:01:45.885694 9471 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [term 3 FOLLOWER]: Leader election won for term 3
I20260929 11:01:45.885807 9471 raft_consensus.cc:686] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [term 3 LEADER]: Becoming Leader. State: Replica: 099a625a94ff4147881504b477a5305e, State: Running, Role: LEADER
I20260929 11:01:45.885902 9467 sys_catalog.cc:564] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:01:45.885897 9471 consensus_queue.cc:237] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "099a625a94ff4147881504b477a5305e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43375 } }
I20260929 11:01:45.886359 9471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 099a625a94ff4147881504b477a5305e. Latest consensus state: current_term: 3 leader_uuid: "099a625a94ff4147881504b477a5305e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "099a625a94ff4147881504b477a5305e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43375 } } }
I20260929 11:01:45.886435 9471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [sys.catalog]: This master's current role is: LEADER
I20260929 11:01:45.886575 9472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "099a625a94ff4147881504b477a5305e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "099a625a94ff4147881504b477a5305e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 43375 } } }
I20260929 11:01:45.886653 9472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 099a625a94ff4147881504b477a5305e [sys.catalog]: This master's current role is: LEADER
I20260929 11:01:45.886767 9476 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:01:45.887585 9476 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:01:45.887686 9476 catalog_manager.cc:1259] Loaded cluster ID: 95defd70f03a4de6b5c5db20a33104a1
I20260929 11:01:45.887710 9476 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260929 11:01:45.887920 9476 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f18d93389ad google::LogMessage::Fail() at ??:0
@ 0x7f18d933b88a google::LogMessage::SendToLog() at ??:0
@ 0x7f18d93384de google::LogMessage::Flush() at ??:0
@ 0x7f18d9339a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f18e12590f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f18e12628b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f18db2060d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f18db1fb954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f18dc05f6db start_thread at ??:0
@ 0x7f18d7e3661f clone at ??:0
W20260929 11:01:45.991127 9469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:01:45.991425 9469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:01:45.991464 9469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:01:45.993177 9469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:01:45.993237 9469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:01:45.993276 9469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.193
I20260929 11:01:45.995430 9469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:33997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.193:37655
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.120.193
--webserver_port=46413
--tserver_master_addrs=127.26.120.254:43375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.193
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:01:45.995723 9469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:01:45.996021 9469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:01:46.001569 9503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:01:46.001600 9504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:01:46.001623 9506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:01:46.001811 9469 server_base.cc:1034] running on GCE node
I20260929 11:01:46.002132 9469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:01:46.002414 9469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:01:46.003574 9469 hybrid_clock.cc:648] HybridClock initialized: now 1790679706003567 us; error 29 us; skew 500 ppm
Sep 29 11:01:46 dist-test-slave-2lzq krb5kdc[8332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679706, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:01:46.006369 9469 init.cc:377] Logged in from keytab as kudu/127.26.120.193@KRBTEST.COM (short username kudu)
I20260929 11:01:46.006841 9469 webserver.cc:460] Webserver started at http://127.26.120.193:46413/ using document root <none> and password file <none>
I20260929 11:01:46.007066 9469 fs_manager.cc:359] Metadata directory not provided
I20260929 11:01:46.007134 9469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:01:46.008652 9469 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 11:01:46.009392 9513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:46.009570 9469 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:01:46.009670 9469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/wal
uuid: "3458dd5df95c4051b0962ec6aa134b41"
format_stamp: "Formatted at 2026-09-29 11:01:44 on dist-test-slave-2lzq"
server_key: "d1b56b76257e6d840e63b9488ef5a1d8"
server_key_iv: "a2d7e63b12db236689e4fdf07978d602"
server_key_version: "encryptionkey@0"
I20260929 11:01:46.010026 9469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:01:46.018796 9469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:01:46.020982 9469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:01:46.021134 9469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:01:46.021359 9469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:01:46.021798 9520 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260929 11:01:46.023777 9469 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 11:01:46.023859 9469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260929 11:01:46.023906 9469 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 11:01:46.024541 9469 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 11:01:46.024601 9469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260929 11:01:46.024668 9520 tablet_bootstrap.cc:492] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41: Bootstrap starting.
I20260929 11:01:46.031947 9469 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.193:37655
I20260929 11:01:46.032092 9627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.193:37655 every 8 connection(s)
I20260929 11:01:46.032379 9469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
I20260929 11:01:46.032666 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 9469
I20260929 11:01:46.032944 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.194:35157
--local_ip_for_outbound_sockets=127.26.120.194
--tserver_master_addrs=127.26.120.254:43375
--webserver_port=36835
--webserver_interface=127.26.120.194
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:33997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 11:01:46.036149 9520 log.cc:824] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41: Log is configured to *not* fsync() on all Append() calls
I20260929 11:01:46.039023 9520 tablet_bootstrap.cc:492] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 11:01:46.039335 9520 tablet_bootstrap.cc:492] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41: Bootstrap complete.
I20260929 11:01:46.040568 9520 ts_tablet_manager.cc:1397] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.009s
I20260929 11:01:46.042011 9520 raft_consensus.cc:348] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6b40d28ccab483f9cda4f705e51e737" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45339 } } peers { permanent_uuid: "3d903e30c836441eab86034bebe7421d" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35157 } } peers { permanent_uuid: "3458dd5df95c4051b0962ec6aa134b41" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37655 } }
I20260929 11:01:46.042353 9520 raft_consensus.cc:729] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3458dd5df95c4051b0962ec6aa134b41, State: Initialized, Role: FOLLOWER
I20260929 11:01:46.042500 9520 consensus_queue.cc:260] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6b40d28ccab483f9cda4f705e51e737" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45339 } } peers { permanent_uuid: "3d903e30c836441eab86034bebe7421d" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35157 } } peers { permanent_uuid: "3458dd5df95c4051b0962ec6aa134b41" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37655 } }
I20260929 11:01:46.042768 9520 ts_tablet_manager.cc:1428] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
W20260929 11:01:46.103272 9631 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 11:01:46.034916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.193:38563)
0929 11:01:46.035143 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:46.035197 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:46.035677 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:46.103051 (+ 67374us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":105,"threads_started":1}
W20260929 11:01:46.103525 9628 heartbeater.cc:644] Failed to heartbeat to 127.26.120.254:43375 (0 consecutive failures): Network error: Failed to ping master at 127.26.120.254:43375: Client connection negotiation failed: client connection to 127.26.120.254:43375: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 11:01:46.103915 9631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:46.103747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.193:58629)
0929 11:01:46.103814 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:46.103875 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260929 11:01:46.104295 9631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:46.104188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.193:55325)
0929 11:01:46.104237 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:46.104268 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260929 11:01:46.104429 9628 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 11:01:46.119011 9630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:01:46.119226 9630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:01:46.119251 9630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:01:46.120500 9630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:01:46.120543 9630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:01:46.120570 9630 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.194
I20260929 11:01:46.122157 9630 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:33997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.194:35157
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.120.194
--webserver_port=36835
--tserver_master_addrs=127.26.120.254:43375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.194
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:01:46.122408 9630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:01:46.122658 9630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:01:46.128109 9638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:01:46.128140 9641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:01:46.128114 9639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:01:46.128333 9630 server_base.cc:1034] running on GCE node
I20260929 11:01:46.128571 9630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:01:46.128826 9630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:01:46.130002 9630 hybrid_clock.cc:648] HybridClock initialized: now 1790679706129991 us; error 28 us; skew 500 ppm
Sep 29 11:01:46 dist-test-slave-2lzq krb5kdc[8332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679706, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:01:46.132617 9630 init.cc:377] Logged in from keytab as kudu/127.26.120.194@KRBTEST.COM (short username kudu)
I20260929 11:01:46.133018 9630 webserver.cc:460] Webserver started at http://127.26.120.194:36835/ using document root <none> and password file <none>
I20260929 11:01:46.133214 9630 fs_manager.cc:359] Metadata directory not provided
I20260929 11:01:46.133256 9630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:01:46.134663 9630 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 11:01:46.135248 9648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:46.135466 9630 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 11:01:46.135556 9630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/wal
uuid: "3d903e30c836441eab86034bebe7421d"
format_stamp: "Formatted at 2026-09-29 11:01:44 on dist-test-slave-2lzq"
server_key: "dbf51f0d0dc37f38ec6b6f2742ae6e7b"
server_key_iv: "d169e87d10e0dd04f2f28eb3c05337ae"
server_key_version: "encryptionkey@0"
I20260929 11:01:46.135908 9630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:01:46.151068 9630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:01:46.153056 9630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:01:46.153179 9630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:01:46.153371 9630 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:01:46.153707 9630 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:01:46.153744 9630 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:46.153792 9630 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:01:46.153829 9630 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:46.160295 9630 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.194:35157
I20260929 11:01:46.160511 9761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.194:35157 every 8 connection(s)
I20260929 11:01:46.160737 9630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
I20260929 11:01:46.162046 9764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:46.161461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.194:51309)
0929 11:01:46.161612 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:46.161690 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":71,"threads_started":1}
W20260929 11:01:46.162304 9762 heartbeater.cc:644] Failed to heartbeat to 127.26.120.254:43375 (0 consecutive failures): Network error: Failed to ping master at 127.26.120.254:43375: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
I20260929 11:01:46.162632 9764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:46.162542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.194:44727)
0929 11:01:46.162588 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:46.162614 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260929 11:01:46.162966 9764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:46.162844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.194:45867)
0929 11:01:46.162912 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:46.162943 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260929 11:01:46.163127 9762 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 11:01:46.169189 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 9630
I20260929 11:01:46.169493 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.195:45339
--local_ip_for_outbound_sockets=127.26.120.195
--tserver_master_addrs=127.26.120.254:43375
--webserver_port=46053
--webserver_interface=127.26.120.195
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:33997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:01:46.251771 9765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:01:46.251982 9765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:01:46.252007 9765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:01:46.253167 9765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:01:46.253211 9765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:01:46.253237 9765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.195
I20260929 11:01:46.254849 9765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:33997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.195:45339
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.120.195
--webserver_port=46053
--tserver_master_addrs=127.26.120.254:43375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.195
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:01:46.255102 9765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:01:46.255349 9765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 11:01:46.260859 9765 server_base.cc:1034] running on GCE node
W20260929 11:01:46.260892 9771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:01:46.260907 9770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:01:46.260842 9773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:01:46.261358 9765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:01:46.261601 9765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:01:46.262741 9765 hybrid_clock.cc:648] HybridClock initialized: now 1790679706262700 us; error 61 us; skew 500 ppm
Sep 29 11:01:46 dist-test-slave-2lzq krb5kdc[8332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679706, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:01:46.265326 9765 init.cc:377] Logged in from keytab as kudu/127.26.120.195@KRBTEST.COM (short username kudu)
I20260929 11:01:46.265790 9765 webserver.cc:460] Webserver started at http://127.26.120.195:46053/ using document root <none> and password file <none>
I20260929 11:01:46.266103 9765 fs_manager.cc:359] Metadata directory not provided
I20260929 11:01:46.266182 9765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:01:46.267628 9765 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:01:46.268308 9780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:46.268515 9765 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 11:01:46.268635 9765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/wal
uuid: "d6b40d28ccab483f9cda4f705e51e737"
format_stamp: "Formatted at 2026-09-29 11:01:44 on dist-test-slave-2lzq"
server_key: "6c622702ddcc679d35bd0e0ac86ab89c"
server_key_iv: "0d84a0d2a688f8ef0207922b61ce0b08"
server_key_version: "encryptionkey@0"
I20260929 11:01:46.269045 9765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:01:46.282490 9765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:01:46.284909 9765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:01:46.285068 9765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:01:46.285305 9765 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:01:46.285755 9787 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260929 11:01:46.287237 9787 ts_tablet_manager.cc:1852] T cfa300a616c5446e820e1732db151f88 P d6b40d28ccab483f9cda4f705e51e737: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260929 11:01:46.287549 9787 ts_tablet_manager.cc:1905] T cfa300a616c5446e820e1732db151f88 P d6b40d28ccab483f9cda4f705e51e737: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 11:01:46.289639 9787 ts_tablet_manager.cc:1918] T cfa300a616c5446e820e1732db151f88 P d6b40d28ccab483f9cda4f705e51e737: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 11:01:46.289746 9787 log.cc:1196] T cfa300a616c5446e820e1732db151f88 P d6b40d28ccab483f9cda4f705e51e737: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/wal/wals/cfa300a616c5446e820e1732db151f88
I20260929 11:01:46.290020 9787 ts_tablet_manager.cc:1939] T cfa300a616c5446e820e1732db151f88 P d6b40d28ccab483f9cda4f705e51e737: Deleting consensus metadata
I20260929 11:01:46.290288 9765 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260929 11:01:46.290351 9765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260929 11:01:46.290410 9765 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:01:46.290442 9765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:01:46.296505 9765 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.195:45339
I20260929 11:01:46.296607 9894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.195:45339 every 8 connection(s)
I20260929 11:01:46.296917 9765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
I20260929 11:01:46.298012 9897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:46.297601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.195:36097)
0929 11:01:46.297759 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:46.297871 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
W20260929 11:01:46.298247 9895 heartbeater.cc:644] Failed to heartbeat to 127.26.120.254:43375 (0 consecutive failures): Network error: Failed to ping master at 127.26.120.254:43375: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
I20260929 11:01:46.298585 9897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:46.298480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.195:59187)
0929 11:01:46.298531 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:46.298556 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260929 11:01:46.298853 9897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:46.298770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.195:34273)
0929 11:01:46.298813 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:46.298835 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260929 11:01:46.298951 9895 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 11:01:46.305176 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 9765
I20260929 11:01:47.105053 9899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:47.104780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.193:36531)
0929 11:01:47.104962 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:47.105004 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":70,"threads_started":1}
I20260929 11:01:47.163785 9900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:47.163472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.194:38601)
0929 11:01:47.163682 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:47.163720 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":56,"threads_started":1}
I20260929 11:01:47.299439 9901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:47.299256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.195:40109)
0929 11:01:47.299373 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:47.299406 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":54,"threads_started":1}
I20260929 11:01:47.799109 9902 raft_consensus.cc:482] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 11:01:47.799239 9902 raft_consensus.cc:504] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6b40d28ccab483f9cda4f705e51e737" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45339 } } peers { permanent_uuid: "3d903e30c836441eab86034bebe7421d" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35157 } } peers { permanent_uuid: "3458dd5df95c4051b0962ec6aa134b41" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37655 } }
I20260929 11:01:47.799566 9902 leader_election.cc:288] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6b40d28ccab483f9cda4f705e51e737 (127.26.120.195:45339), 3d903e30c836441eab86034bebe7421d (127.26.120.194:35157)
Sep 29 11:01:47 dist-test-slave-2lzq krb5kdc[8332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679706, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for kudu/127.26.120.194@KRBTEST.COM
Sep 29 11:01:47 dist-test-slave-2lzq krb5kdc[8332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679706, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for kudu/127.26.120.195@KRBTEST.COM
I20260929 11:01:47.812355 9903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:47.799739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:35157 (local address 127.26.120.193:49083)
0929 11:01:47.799938 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:47.799947 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:47.799989 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:47.800818 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:47.800821 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:47.801355 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:01:47.801627 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:47.801635 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:47.803337 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:47.803338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:47.804610 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:47.804613 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:47.805469 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:47.805470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:47.805618 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:47.806335 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:01:47.806356 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:01:47.808077 (+ 1721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:01:47.810480 (+ 2403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:47.810487 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:47.810489 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:47.810805 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:47.811122 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:47.811123 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:47.811124 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:47.811249 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:47.811767 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:01:47.811768 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:01:47.811944 (+ 176us) client_negotiation.cc:769] Sending connection context
0929 11:01:47.812063 (+ 119us) client_negotiation.cc:240] Negotiation successful
0929 11:01:47.812261 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":71,"thread_start_us":51,"threads_started":1}
I20260929 11:01:47.812366 9905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:47.799828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:35897 (local address 127.26.120.195:45339)
0929 11:01:47.800004 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:47.800035 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:47.800059 (+ 24us) server_negotiation.cc:407] Connection header received
0929 11:01:47.800628 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:47.800631 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:47.800684 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:47.800849 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:01:47.801753 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:47.803180 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:47.804708 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:47.805343 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:47.805461 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:47.808562 (+ 3101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:01:47.808566 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:01:47.808567 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:01:47.808578 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:01:47.810599 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:47.810992 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:47.810994 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:47.810995 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:47.811098 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:47.811322 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:47.811323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:47.811323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:47.811665 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:01:47.811762 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:47.812130 (+ 368us) server_negotiation.cc:299] Negotiation successful
0929 11:01:47.812249 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":61,"threads_started":1}
I20260929 11:01:47.812427 9906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:47.799925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:49083 (local address 127.26.120.194:35157)
0929 11:01:47.800067 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0929 11:01:47.800075 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 11:01:47.800092 (+ 17us) server_negotiation.cc:407] Connection header received
0929 11:01:47.800580 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:01:47.800582 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:01:47.800626 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:01:47.800783 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:01:47.801754 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:47.803180 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:47.804708 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:01:47.805343 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:01:47.805463 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:47.808195 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:01:47.808199 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:01:47.808200 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:01:47.808218 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:01:47.810398 (+ 2180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:47.810900 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:47.810908 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:47.810908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:47.811037 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:01:47.811323 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:01:47.811324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:01:47.811324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:01:47.811677 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:01:47.811774 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 11:01:47.812152 (+ 378us) server_negotiation.cc:299] Negotiation successful
0929 11:01:47.812310 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":49,"threads_started":1}
I20260929 11:01:47.812466 9904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:47.799735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:45339 (local address 127.26.120.193:35897)
0929 11:01:47.799859 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:47.799870 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 11:01:47.799930 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:01:47.800838 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:01:47.800839 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:01:47.801355 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:01:47.801627 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:47.801635 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:47.803307 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:47.803308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:47.804610 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:01:47.804613 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:01:47.805467 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:01:47.805469 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:01:47.805618 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:01:47.806335 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:01:47.806356 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:01:47.808458 (+ 2102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:01:47.810676 (+ 2218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:47.810677 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:47.810678 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:47.810912 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:47.811165 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:01:47.811165 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:01:47.811166 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:01:47.811249 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:01:47.811745 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:01:47.811752 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:01:47.811941 (+ 189us) client_negotiation.cc:769] Sending connection context
0929 11:01:47.812062 (+ 121us) client_negotiation.cc:240] Negotiation successful
0929 11:01:47.812341 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":219,"thread_start_us":48,"threads_started":1}
W20260929 11:01:47.813499 9516 leader_election.cc:341] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d6b40d28ccab483f9cda4f705e51e737 (127.26.120.195:45339): Not found: Tablet not found: cfa300a616c5446e820e1732db151f88
W20260929 11:01:47.813589 9515 leader_election.cc:341] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3d903e30c836441eab86034bebe7421d (127.26.120.194:35157): Not found: Tablet not found: cfa300a616c5446e820e1732db151f88
I20260929 11:01:47.813642 9515 leader_election.cc:302] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3458dd5df95c4051b0962ec6aa134b41; no voters: 3d903e30c836441eab86034bebe7421d, d6b40d28ccab483f9cda4f705e51e737
I20260929 11:01:47.813751 9902 raft_consensus.cc:2726] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 11:01:48.105712 9904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:48.105574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.193:48665)
0929 11:01:48.105640 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:48.105673 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260929 11:01:48.164533 9907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:48.164316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.194:40437)
0929 11:01:48.164433 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:48.164489 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":60,"threads_started":1}
I20260929 11:01:48.300143 9908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:48.299935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.195:54235)
0929 11:01:48.300064 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:48.300105 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":61,"threads_started":1}
I20260929 11:01:49.106467 9909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:49.106256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.193:52289)
0929 11:01:49.106379 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:49.106417 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":60,"threads_started":1}
I20260929 11:01:49.165211 9910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:49.165006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.194:51695)
0929 11:01:49.165133 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:49.165171 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":62,"threads_started":1}
I20260929 11:01:49.300788 9911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:49.300594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.195:38705)
0929 11:01:49.300710 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:49.300748 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":61,"threads_started":1}
I20260929 11:01:49.756594 9912 raft_consensus.cc:482] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 11:01:49.756697 9912 raft_consensus.cc:504] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6b40d28ccab483f9cda4f705e51e737" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45339 } } peers { permanent_uuid: "3d903e30c836441eab86034bebe7421d" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35157 } } peers { permanent_uuid: "3458dd5df95c4051b0962ec6aa134b41" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37655 } }
I20260929 11:01:49.756933 9912 leader_election.cc:288] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6b40d28ccab483f9cda4f705e51e737 (127.26.120.195:45339), 3d903e30c836441eab86034bebe7421d (127.26.120.194:35157)
W20260929 11:01:49.757416 9516 leader_election.cc:341] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d6b40d28ccab483f9cda4f705e51e737 (127.26.120.195:45339): Not found: Tablet not found: cfa300a616c5446e820e1732db151f88
W20260929 11:01:49.757498 9515 leader_election.cc:341] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3d903e30c836441eab86034bebe7421d (127.26.120.194:35157): Not found: Tablet not found: cfa300a616c5446e820e1732db151f88
I20260929 11:01:49.757588 9515 leader_election.cc:302] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3458dd5df95c4051b0962ec6aa134b41; no voters: 3d903e30c836441eab86034bebe7421d, d6b40d28ccab483f9cda4f705e51e737
I20260929 11:01:49.757686 9912 raft_consensus.cc:2726] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 11:01:50.107232 9913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:50.107022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.193:35313)
0929 11:01:50.107153 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:50.107186 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":78,"threads_started":1}
I20260929 11:01:50.165952 9914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:50.165710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.194:58399)
0929 11:01:50.165848 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:50.165889 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":71,"threads_started":1}
I20260929 11:01:50.301589 9915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:50.301383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.195:32937)
0929 11:01:50.301507 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:50.301548 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":59,"threads_started":1}
I20260929 11:01:51.107990 9916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:51.107771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.193:39311)
0929 11:01:51.107901 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:51.107941 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":82,"threads_started":1}
I20260929 11:01:51.166708 9917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:51.166508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.194:35813)
0929 11:01:51.166625 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:51.166665 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":57,"threads_started":1}
I20260929 11:01:51.302289 9918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:51.302078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.195:54145)
0929 11:01:51.302201 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:51.302246 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":57,"threads_started":1}
I20260929 11:01:52.108732 9919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:52.108482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.193:51105)
0929 11:01:52.108642 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:52.108687 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":78,"threads_started":1}
I20260929 11:01:52.167380 9920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:52.167158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.194:37837)
0929 11:01:52.167287 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:52.167339 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":72,"threads_started":1}
I20260929 11:01:52.303017 9921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:52.302795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.195:42377)
0929 11:01:52.302931 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:52.302971 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":65,"threads_started":1}
I20260929 11:01:53.050244 9922 raft_consensus.cc:482] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 11:01:53.050382 9922 raft_consensus.cc:504] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6b40d28ccab483f9cda4f705e51e737" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45339 } } peers { permanent_uuid: "3d903e30c836441eab86034bebe7421d" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35157 } } peers { permanent_uuid: "3458dd5df95c4051b0962ec6aa134b41" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37655 } }
I20260929 11:01:53.050601 9922 leader_election.cc:288] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6b40d28ccab483f9cda4f705e51e737 (127.26.120.195:45339), 3d903e30c836441eab86034bebe7421d (127.26.120.194:35157)
W20260929 11:01:53.051041 9516 leader_election.cc:341] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d6b40d28ccab483f9cda4f705e51e737 (127.26.120.195:45339): Not found: Tablet not found: cfa300a616c5446e820e1732db151f88
W20260929 11:01:53.051121 9515 leader_election.cc:341] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3d903e30c836441eab86034bebe7421d (127.26.120.194:35157): Not found: Tablet not found: cfa300a616c5446e820e1732db151f88
I20260929 11:01:53.051156 9515 leader_election.cc:302] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3458dd5df95c4051b0962ec6aa134b41; no voters: 3d903e30c836441eab86034bebe7421d, d6b40d28ccab483f9cda4f705e51e737
I20260929 11:01:53.051267 9922 raft_consensus.cc:2726] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 11:01:53.109543 9923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:53.109289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.193:55777)
0929 11:01:53.109454 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:53.109496 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":82,"threads_started":1}
I20260929 11:01:53.168093 9924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:53.167847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.194:48853)
0929 11:01:53.168001 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:53.168038 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":75,"threads_started":1}
I20260929 11:01:53.303692 9925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:53.303470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.195:56971)
0929 11:01:53.303614 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:53.303652 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":62,"threads_started":1}
I20260929 11:01:54.110348 9926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:54.110107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.193:37409)
0929 11:01:54.110259 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:54.110303 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":68,"threads_started":1}
I20260929 11:01:54.168883 9927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:54.168638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.194:53667)
0929 11:01:54.168799 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:54.168838 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":75,"threads_started":1}
I20260929 11:01:54.304395 9928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:54.304167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.195:41365)
0929 11:01:54.304317 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:54.304355 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":76,"threads_started":1}
I20260929 11:01:55.111142 9929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:55.110915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.193:50389)
0929 11:01:55.111056 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:55.111095 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":60,"threads_started":1}
I20260929 11:01:55.169740 9930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:55.169525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.194:34677)
0929 11:01:55.169656 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:55.169697 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":58,"threads_started":1}
I20260929 11:01:55.305090 9931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:55.304877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.195:33527)
0929 11:01:55.305014 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:55.305051 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":76,"threads_started":1}
I20260929 11:01:56.092244 9932 raft_consensus.cc:482] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 11:01:56.092341 9932 raft_consensus.cc:504] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6b40d28ccab483f9cda4f705e51e737" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45339 } } peers { permanent_uuid: "3d903e30c836441eab86034bebe7421d" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35157 } } peers { permanent_uuid: "3458dd5df95c4051b0962ec6aa134b41" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37655 } }
I20260929 11:01:56.092660 9932 leader_election.cc:288] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6b40d28ccab483f9cda4f705e51e737 (127.26.120.195:45339), 3d903e30c836441eab86034bebe7421d (127.26.120.194:35157)
W20260929 11:01:56.093173 9515 leader_election.cc:341] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3d903e30c836441eab86034bebe7421d (127.26.120.194:35157): Not found: Tablet not found: cfa300a616c5446e820e1732db151f88
W20260929 11:01:56.093256 9516 leader_election.cc:341] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d6b40d28ccab483f9cda4f705e51e737 (127.26.120.195:45339): Not found: Tablet not found: cfa300a616c5446e820e1732db151f88
I20260929 11:01:56.093353 9516 leader_election.cc:302] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3458dd5df95c4051b0962ec6aa134b41; no voters: 3d903e30c836441eab86034bebe7421d, d6b40d28ccab483f9cda4f705e51e737
I20260929 11:01:56.093429 9932 raft_consensus.cc:2726] T cfa300a616c5446e820e1732db151f88 P 3458dd5df95c4051b0962ec6aa134b41 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 11:01:56.111788 9933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:56.111591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.193:52601)
0929 11:01:56.111707 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:56.111741 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":63,"threads_started":1}
I20260929 11:01:56.170518 9934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:56.170283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.194:33447)
0929 11:01:56.170426 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:56.170471 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":83,"threads_started":1}
I20260929 11:01:56.305800 9935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:56.305585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.195:48757)
0929 11:01:56.305715 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:56.305756 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":69,"threads_started":1}
I20260929 11:01:57.112561 9936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:57.112289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.193:53385)
0929 11:01:57.112449 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:57.112495 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":73,"threads_started":1}
I20260929 11:01:57.171382 9937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:57.171096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.194:34195)
0929 11:01:57.171278 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:57.171333 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":62,"threads_started":1}
I20260929 11:01:57.306577 9938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:57.306313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.195:38933)
0929 11:01:57.306489 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:57.306532 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":76,"threads_started":1}
I20260929 11:01:58.113291 9939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:58.113032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.193:34231)
0929 11:01:58.113202 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:58.113243 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":89,"threads_started":1}
I20260929 11:01:58.172418 9940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:58.172051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.194:52455)
0929 11:01:58.172258 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:58.172304 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":113,"threads_started":1}
I20260929 11:01:58.307404 9941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:58.307169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.195:35503)
0929 11:01:58.307315 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:58.307358 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":75,"threads_started":1}
I20260929 11:01:59.114142 9942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:59.113827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.193:37429)
0929 11:01:59.114009 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:59.114043 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":124,"threads_started":1}
I20260929 11:01:59.173162 9943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:59.172935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.194:43085)
0929 11:01:59.173073 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:59.173116 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":64,"threads_started":1}
I20260929 11:01:59.308137 9944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:01:59.307906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.195:44931)
0929 11:01:59.308053 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0929 11:01:59.308092 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":60,"threads_started":1}
I20260929 11:02:00.114825 9945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:00.114585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.193:38195)
0929 11:02:00.114728 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:00.114764 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":76,"threads_started":1}
I20260929 11:02:00.173897 9946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:00.173653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.194:47021)
0929 11:02:00.173810 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:00.173849 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":88,"threads_started":1}
I20260929 11:02:00.308914 9947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:00.308662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.195:59593)
0929 11:02:00.308828 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:00.308869 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":84,"threads_started":1}
I20260929 11:02:01.115726 9948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:01.115499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.193:43535)
0929 11:02:01.115638 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:01.115680 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":64,"threads_started":1}
I20260929 11:02:01.174698 9949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:01.174458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.194:44729)
0929 11:02:01.174608 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:01.174652 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":66,"threads_started":1}
I20260929 11:02:01.309733 9950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:01.309502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:43375 (local address 127.26.120.195:46229)
0929 11:02:01.309641 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:01.309685 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":64,"threads_started":1}
I20260929 11:02:01.314976 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 9469
I20260929 11:02:01.319981 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 9630
I20260929 11:02:01.324491 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 9765
2026-09-29T11:02:01Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18224 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 11:02:01 dist-test-slave-2lzq krb5kdc[9953](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 11:02:01 dist-test-slave-2lzq krb5kdc[9953](info): set up 2 sockets
Sep 29 11:02:01 dist-test-slave-2lzq krb5kdc[9953](info): commencing operation
krb5kdc: starting...
W20260929 11:02:02.358495 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 11:02:02 dist-test-slave-2lzq krb5kdc[9953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T11:02:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T11:02:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.120.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
I20260929 11:02:02.484663 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:44319
--webserver_interface=127.26.120.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:34649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:44319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:02:02.566627 9966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:02:02.566843 9966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:02:02.566865 9966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:02:02.568073 9966 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 11:02:02.568115 9966 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 11:02:02.568130 9966 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 11:02:02.568140 9966 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:02:02.568190 9966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:02:02.568216 9966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:02:02.568239 9966 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:02:02.568267 9966 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:02:02.569881 9966 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:34649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:44319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:44319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:02:02.570174 9966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:02:02.570433 9966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 11:02:02.576022 9966 server_base.cc:1034] running on GCE node
W20260929 11:02:02.575994 9972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:02:02.576031 9971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:02:02.575987 9974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:02:02.576579 9966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:02:02.576887 9966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:02:02.578132 9966 hybrid_clock.cc:648] HybridClock initialized: now 1790679722578111 us; error 30 us; skew 500 ppm
Sep 29 11:02:02 dist-test-slave-2lzq krb5kdc[9953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679722, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:02:02.580701 9966 init.cc:377] Logged in from keytab as kudu/127.26.120.254@KRBTEST.COM (short username kudu)
I20260929 11:02:02.581094 9966 webserver.cc:460] Webserver started at http://127.26.120.254:36475/ using document root <none> and password file <none>
I20260929 11:02:02.581355 9966 fs_manager.cc:359] Metadata directory not provided
I20260929 11:02:02.581403 9966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:02:02.581537 9966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:02:02.582677 9966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "037fbc5362fc4c5ab9f7ece66813ebe4"
format_stamp: "Formatted at 2026-09-29 11:02:02 on dist-test-slave-2lzq"
server_key: "307b583f309db733d33df0a955b7cbc0"
server_key_iv: "e6a04accb7f7dddea1607b03bbd44190"
server_key_version: "encryptionkey@0"
I20260929 11:02:02.583038 9966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "037fbc5362fc4c5ab9f7ece66813ebe4"
format_stamp: "Formatted at 2026-09-29 11:02:02 on dist-test-slave-2lzq"
server_key: "307b583f309db733d33df0a955b7cbc0"
server_key_iv: "e6a04accb7f7dddea1607b03bbd44190"
server_key_version: "encryptionkey@0"
I20260929 11:02:02.584450 9966 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 11:02:02.585280 9981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:02:02.585572 9966 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:02:02.585676 9966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "037fbc5362fc4c5ab9f7ece66813ebe4"
format_stamp: "Formatted at 2026-09-29 11:02:02 on dist-test-slave-2lzq"
server_key: "307b583f309db733d33df0a955b7cbc0"
server_key_iv: "e6a04accb7f7dddea1607b03bbd44190"
server_key_version: "encryptionkey@0"
I20260929 11:02:02.585764 9966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:02:02.593482 9966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:02:02.595631 9966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:02:02.595742 9966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:02:02.599920 9966 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:44319
I20260929 11:02:02.599983 10033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:44319 every 8 connection(s)
I20260929 11:02:02.600302 9966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:02:02.601148 10034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:02:02.603542 10034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4: Bootstrap starting.
I20260929 11:02:02.604143 10034 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4: Neither blocks nor log segments found. Creating new log.
I20260929 11:02:02.604444 10034 log.cc:824] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4: Log is configured to *not* fsync() on all Append() calls
I20260929 11:02:02.605105 10034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4: No bootstrap required, opened a new log
I20260929 11:02:02.606690 10034 raft_consensus.cc:348] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "037fbc5362fc4c5ab9f7ece66813ebe4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44319 } }
I20260929 11:02:02.606853 10034 raft_consensus.cc:374] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:02:02.606901 10034 raft_consensus.cc:729] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 037fbc5362fc4c5ab9f7ece66813ebe4, State: Initialized, Role: FOLLOWER
I20260929 11:02:02.607049 10034 consensus_queue.cc:260] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "037fbc5362fc4c5ab9f7ece66813ebe4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44319 } }
I20260929 11:02:02.607111 10034 raft_consensus.cc:388] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 11:02:02.607132 10034 raft_consensus.cc:482] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 11:02:02.607203 10034 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:02:02.607728 10034 raft_consensus.cc:504] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "037fbc5362fc4c5ab9f7ece66813ebe4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44319 } }
I20260929 11:02:02.607846 10034 leader_election.cc:302] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 037fbc5362fc4c5ab9f7ece66813ebe4; no voters:
I20260929 11:02:02.608027 10034 leader_election.cc:288] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 11:02:02.608055 10037 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:02:02.608188 10037 raft_consensus.cc:686] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [term 1 LEADER]: Becoming Leader. State: Replica: 037fbc5362fc4c5ab9f7ece66813ebe4, State: Running, Role: LEADER
I20260929 11:02:02.608305 10034 sys_catalog.cc:564] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:02:02.608345 10037 consensus_queue.cc:237] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "037fbc5362fc4c5ab9f7ece66813ebe4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44319 } }
I20260929 11:02:02.609037 10038 sys_catalog.cc:455] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "037fbc5362fc4c5ab9f7ece66813ebe4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "037fbc5362fc4c5ab9f7ece66813ebe4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44319 } } }
I20260929 11:02:02.609037 10039 sys_catalog.cc:455] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 037fbc5362fc4c5ab9f7ece66813ebe4. Latest consensus state: current_term: 1 leader_uuid: "037fbc5362fc4c5ab9f7ece66813ebe4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "037fbc5362fc4c5ab9f7ece66813ebe4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44319 } } }
I20260929 11:02:02.609159 10038 sys_catalog.cc:458] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [sys.catalog]: This master's current role is: LEADER
I20260929 11:02:02.609165 10039 sys_catalog.cc:458] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [sys.catalog]: This master's current role is: LEADER
I20260929 11:02:02.609400 10046 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:02:02.609720 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 9966
I20260929 11:02:02.609822 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 11:02:02.610100 27107 external_mini_cluster.cc:1428] Setting key 1a5172151ab79d19f917da837f9de1ea
I20260929 11:02:02.610576 10046 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:02:02.612190 10046 catalog_manager.cc:1347] Generated new cluster ID: 61e7e82bb0c448e3b54e7fcdfde61c94
I20260929 11:02:02.612243 10046 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 11:02:02 dist-test-slave-2lzq krb5kdc[9953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:02:02.622030 10056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:02.610604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46148 (local address 127.26.120.254:44319)
0929 11:02:02.610779 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0929 11:02:02.610801 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0929 11:02:02.610836 (+ 35us) server_negotiation.cc:407] Connection header received
0929 11:02:02.610957 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:02:02.610965 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:02:02.611020 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:02:02.611177 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:02:02.611899 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:02.613293 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:02.614433 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:02.615050 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:02.615146 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:02.618114 (+ 2968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:02:02.618119 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:02:02.618119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:02:02.618132 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:02:02.620191 (+ 2059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:02:02.620583 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:02:02.620587 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:02:02.620587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:02:02.620704 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:02:02.620914 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:02:02.620914 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:02:02.620915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:02:02.621267 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:02:02.621431 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0929 11:02:02.621533 (+ 102us) server_negotiation.cc:299] Negotiation successful
0929 11:02:02.621784 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":76,"threads_started":1}
I20260929 11:02:02.627777 10046 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 11:02:02.628571 10046 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:02:02.631102 10046 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4: Generated new TSK 0
I20260929 11:02:02.631282 10046 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.120.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab.
Entry for principal kudu/127.26.120.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab.
I20260929 11:02:02.660698 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.193:0
--local_ip_for_outbound_sockets=127.26.120.193
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:44319
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:34649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:02:02.746640 10060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:02:02.746848 10060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:02:02.746872 10060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:02:02.748026 10060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:02:02.748066 10060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:02:02.748093 10060 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.193
I20260929 11:02:02.749531 10060 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:34649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:44319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.193
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:02:02.749769 10060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:02:02.750011 10060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:02:02.755158 10066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:02:02.755115 10068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:02:02.755115 10065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:02:02.755548 10060 server_base.cc:1034] running on GCE node
I20260929 11:02:02.755745 10060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:02:02.756007 10060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:02:02.757154 10060 hybrid_clock.cc:648] HybridClock initialized: now 1790679722757142 us; error 28 us; skew 500 ppm
Sep 29 11:02:02 dist-test-slave-2lzq krb5kdc[9953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679722, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:02:02.759644 10060 init.cc:377] Logged in from keytab as kudu/127.26.120.193@KRBTEST.COM (short username kudu)
I20260929 11:02:02.760070 10060 webserver.cc:460] Webserver started at http://127.26.120.193:44249/ using document root <none> and password file <none>
I20260929 11:02:02.760295 10060 fs_manager.cc:359] Metadata directory not provided
I20260929 11:02:02.760341 10060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:02:02.760514 10060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:02:02.761566 10060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/data/instance:
uuid: "20585d017b4b4bfb85a733ae593ebbeb"
format_stamp: "Formatted at 2026-09-29 11:02:02 on dist-test-slave-2lzq"
server_key: "07c5bcb871d78c26fe0da942254088ac"
server_key_iv: "fb426bbcd89a9cdb68356ee7b1ba8e0c"
server_key_version: "encryptionkey@0"
I20260929 11:02:02.761924 10060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance:
uuid: "20585d017b4b4bfb85a733ae593ebbeb"
format_stamp: "Formatted at 2026-09-29 11:02:02 on dist-test-slave-2lzq"
server_key: "07c5bcb871d78c26fe0da942254088ac"
server_key_iv: "fb426bbcd89a9cdb68356ee7b1ba8e0c"
server_key_version: "encryptionkey@0"
I20260929 11:02:02.763269 10060 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:02:02.764078 10075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:02:02.764256 10060 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:02:02.764359 10060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/wal
uuid: "20585d017b4b4bfb85a733ae593ebbeb"
format_stamp: "Formatted at 2026-09-29 11:02:02 on dist-test-slave-2lzq"
server_key: "07c5bcb871d78c26fe0da942254088ac"
server_key_iv: "fb426bbcd89a9cdb68356ee7b1ba8e0c"
server_key_version: "encryptionkey@0"
I20260929 11:02:02.764448 10060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:02:02.781917 10060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:02:02.783829 10060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:02:02.784008 10060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:02:02.784257 10060 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:02:02.784571 10060 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:02:02.784626 10060 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:02:02.784672 10060 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:02:02.784703 10060 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:02:02.790658 10060 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.193:44527
I20260929 11:02:02.790707 10188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.193:44527 every 8 connection(s)
I20260929 11:02:02.791054 10060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
I20260929 11:02:02.796066 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 10060
I20260929 11:02:02.796197 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance
I20260929 11:02:02.796404 27107 external_mini_cluster.cc:1428] Setting key 2def96925bfda60cd42783680f6aa286
Sep 29 11:02:02 dist-test-slave-2lzq krb5kdc[9953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679722, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:02:02.801077 10056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:02.791754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:55071 (local address 127.26.120.254:44319)
0929 11:02:02.791819 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 11:02:02.791821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:02:02.792046 (+ 225us) server_negotiation.cc:407] Connection header received
0929 11:02:02.792553 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:02:02.792554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:02:02.792566 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:02:02.792628 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:02:02.793589 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:02.794111 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:02.795109 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:02.795312 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:02.795365 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:02.797997 (+ 2632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:02:02.797999 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:02:02.797999 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:02:02.798006 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:02:02.799559 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:02:02.800028 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:02:02.800029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:02:02.800030 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:02:02.800095 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:02:02.800335 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:02:02.800336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:02:02.800337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:02:02.800494 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:02:02.800566 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 11:02:02.800823 (+ 257us) server_negotiation.cc:299] Negotiation successful
0929 11:02:02.800935 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 11:02:02.801097 10191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:02.791736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.193:55071)
0929 11:02:02.791906 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:02.791949 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 11:02:02.792464 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:02:02.792713 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:02:02.792717 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:02:02.793277 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:02:02.793500 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:02.793505 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:02.794194 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:02.794195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:02.795025 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:02.795028 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:02.795385 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:02.795385 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:02.795508 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:02.796032 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:02:02.796039 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:02:02.797889 (+ 1850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:02:02.799639 (+ 1750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:02:02.799646 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:02:02.799648 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:02:02.799943 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:02:02.800168 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:02:02.800169 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:02:02.800169 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:02:02.800270 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:02:02.800560 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:02:02.800564 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:02:02.800680 (+ 116us) client_negotiation.cc:769] Sending connection context
0929 11:02:02.800769 (+ 89us) client_negotiation.cc:240] Negotiation successful
0929 11:02:02.800889 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":97,"threads_started":1}
I20260929 11:02:02.801705 10189 heartbeater.cc:347] Connected to a master server at 127.26.120.254:44319
I20260929 11:02:02.801815 10189 heartbeater.cc:464] Registering TS with master...
I20260929 11:02:02.802021 10189 heartbeater.cc:511] Master 127.26.120.254:44319 requested a full tablet report, sending...
I20260929 11:02:02.802670 9998 ts_manager.cc:194] Registered new tserver with Master: 20585d017b4b4bfb85a733ae593ebbeb (127.26.120.193:44527)
I20260929 11:02:02.803679 9998 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.193@KRBTEST.COM'} at 127.26.120.193:55071
WARNING: no policy specified for kudu/127.26.120.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab.
Entry for principal kudu/127.26.120.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab.
I20260929 11:02:02.824823 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.194:0
--local_ip_for_outbound_sockets=127.26.120.194
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:44319
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:34649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:02:02.904923 10194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:02:02.905117 10194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:02:02.905139 10194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:02:02.906311 10194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:02:02.906349 10194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:02:02.906374 10194 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.194
I20260929 11:02:02.907868 10194 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:34649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:44319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.194
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:02:02.908102 10194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:02:02.908334 10194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:02:02.913334 10202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:02:02.913452 10194 server_base.cc:1034] running on GCE node
W20260929 11:02:02.913335 10199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:02:02.913420 10200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:02:02.913738 10194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:02:02.913993 10194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:02:02.915161 10194 hybrid_clock.cc:648] HybridClock initialized: now 1790679722915138 us; error 40 us; skew 500 ppm
Sep 29 11:02:02 dist-test-slave-2lzq krb5kdc[9953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679722, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:02:02.917608 10194 init.cc:377] Logged in from keytab as kudu/127.26.120.194@KRBTEST.COM (short username kudu)
I20260929 11:02:02.918126 10194 webserver.cc:460] Webserver started at http://127.26.120.194:40263/ using document root <none> and password file <none>
I20260929 11:02:02.918345 10194 fs_manager.cc:359] Metadata directory not provided
I20260929 11:02:02.918411 10194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:02:02.918566 10194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:02:02.919585 10194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/data/instance:
uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074"
format_stamp: "Formatted at 2026-09-29 11:02:02 on dist-test-slave-2lzq"
server_key: "f9923899c0f03499f5bb1ff248a5c5e6"
server_key_iv: "28668a3585e22ba56b831a649a0e6206"
server_key_version: "encryptionkey@0"
I20260929 11:02:02.919946 10194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance:
uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074"
format_stamp: "Formatted at 2026-09-29 11:02:02 on dist-test-slave-2lzq"
server_key: "f9923899c0f03499f5bb1ff248a5c5e6"
server_key_iv: "28668a3585e22ba56b831a649a0e6206"
server_key_version: "encryptionkey@0"
I20260929 11:02:02.921288 10194 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 11:02:02.922174 10209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:02:02.922356 10194 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 11:02:02.922453 10194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/wal
uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074"
format_stamp: "Formatted at 2026-09-29 11:02:02 on dist-test-slave-2lzq"
server_key: "f9923899c0f03499f5bb1ff248a5c5e6"
server_key_iv: "28668a3585e22ba56b831a649a0e6206"
server_key_version: "encryptionkey@0"
I20260929 11:02:02.922541 10194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:02:02.935560 10194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:02:02.937521 10194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:02:02.937700 10194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:02:02.937948 10194 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:02:02.938346 10194 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:02:02.938403 10194 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:02:02.938467 10194 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:02:02.938508 10194 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:02:02.944311 10194 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.194:35161
I20260929 11:02:02.944351 10322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.194:35161 every 8 connection(s)
I20260929 11:02:02.944702 10194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
I20260929 11:02:02.949914 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 10194
I20260929 11:02:02.950047 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance
I20260929 11:02:02.950254 27107 external_mini_cluster.cc:1428] Setting key d3b812b3eada1eb3df9135d8628fefcc
Sep 29 11:02:02 dist-test-slave-2lzq krb5kdc[9953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679722, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:02:02.954126 10056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:02.945410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:47845 (local address 127.26.120.254:44319)
0929 11:02:02.945457 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 11:02:02.945459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:02:02.945665 (+ 206us) server_negotiation.cc:407] Connection header received
0929 11:02:02.946146 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:02:02.946147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:02:02.946157 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:02:02.946216 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:02:02.947094 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:02.947619 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:02.948589 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:02.948779 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:02.948837 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:02.951170 (+ 2333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:02:02.951172 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:02:02.951172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:02:02.951180 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:02:02.952651 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:02:02.953081 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:02:02.953082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:02:02.953082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:02:02.953144 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:02:02.953407 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:02:02.953408 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:02:02.953409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:02:02.953525 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:02:02.953601 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 11:02:02.953905 (+ 304us) server_negotiation.cc:299] Negotiation successful
0929 11:02:02.953988 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 11:02:02.954337 10325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:02.945392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.194:47845)
0929 11:02:02.945547 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:02.945581 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 11:02:02.946036 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:02:02.946261 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:02:02.946264 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:02:02.946820 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:02:02.947011 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:02.947015 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:02.947706 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:02.947707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:02.948516 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:02.948518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:02.948884 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:02.948885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:02.948996 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:02.949488 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:02:02.949496 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:02:02.951070 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:02:02.952753 (+ 1683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:02:02.952760 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:02:02.952763 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:02:02.952994 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:02:02.953213 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:02:02.953214 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:02:02.953214 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:02:02.953332 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:02:02.953592 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:02:02.953598 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:02:02.953737 (+ 139us) client_negotiation.cc:769] Sending connection context
0929 11:02:02.953883 (+ 146us) client_negotiation.cc:240] Negotiation successful
0929 11:02:02.954044 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":85,"threads_started":1}
I20260929 11:02:02.954975 10323 heartbeater.cc:347] Connected to a master server at 127.26.120.254:44319
I20260929 11:02:02.955084 10323 heartbeater.cc:464] Registering TS with master...
I20260929 11:02:02.955291 10323 heartbeater.cc:511] Master 127.26.120.254:44319 requested a full tablet report, sending...
I20260929 11:02:02.955699 9998 ts_manager.cc:194] Registered new tserver with Master: 0d9c8dbb6e1e49a59e61c94c6c79e074 (127.26.120.194:35161)
I20260929 11:02:02.956064 9998 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.194@KRBTEST.COM'} at 127.26.120.194:47845
WARNING: no policy specified for kudu/127.26.120.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab.
Entry for principal kudu/127.26.120.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab.
I20260929 11:02:02.976786 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.195:0
--local_ip_for_outbound_sockets=127.26.120.195
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:44319
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:34649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:02:03.057924 10328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:02:03.058192 10328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:02:03.058233 10328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:02:03.059388 10328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:02:03.059469 10328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:02:03.059514 10328 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.195
I20260929 11:02:03.060993 10328 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:34649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:44319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.195
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:02:03.061261 10328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:02:03.061496 10328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:02:03.066481 10336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:02:03.066466 10334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:02:03.066552 10328 server_base.cc:1034] running on GCE node
W20260929 11:02:03.066550 10333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:02:03.066949 10328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:02:03.067191 10328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:02:03.068334 10328 hybrid_clock.cc:648] HybridClock initialized: now 1790679723068320 us; error 31 us; skew 500 ppm
Sep 29 11:02:03 dist-test-slave-2lzq krb5kdc[9953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679723, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:02:03.076498 10328 init.cc:377] Logged in from keytab as kudu/127.26.120.195@KRBTEST.COM (short username kudu)
I20260929 11:02:03.077023 10328 webserver.cc:460] Webserver started at http://127.26.120.195:41745/ using document root <none> and password file <none>
I20260929 11:02:03.077301 10328 fs_manager.cc:359] Metadata directory not provided
I20260929 11:02:03.077361 10328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:02:03.077558 10328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:02:03.078770 10328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/data/instance:
uuid: "46f46a759dfc42838abffdd16b09d2d7"
format_stamp: "Formatted at 2026-09-29 11:02:03 on dist-test-slave-2lzq"
server_key: "99ef178a55e0037763e2479557d74d62"
server_key_iv: "bc24c09cd7a2b69055dcf9ad767d3706"
server_key_version: "encryptionkey@0"
I20260929 11:02:03.079165 10328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance:
uuid: "46f46a759dfc42838abffdd16b09d2d7"
format_stamp: "Formatted at 2026-09-29 11:02:03 on dist-test-slave-2lzq"
server_key: "99ef178a55e0037763e2479557d74d62"
server_key_iv: "bc24c09cd7a2b69055dcf9ad767d3706"
server_key_version: "encryptionkey@0"
I20260929 11:02:03.080585 10328 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 11:02:03.081388 10343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:02:03.081590 10328 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:02:03.081668 10328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/wal
uuid: "46f46a759dfc42838abffdd16b09d2d7"
format_stamp: "Formatted at 2026-09-29 11:02:03 on dist-test-slave-2lzq"
server_key: "99ef178a55e0037763e2479557d74d62"
server_key_iv: "bc24c09cd7a2b69055dcf9ad767d3706"
server_key_version: "encryptionkey@0"
I20260929 11:02:03.081768 10328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:02:03.108608 10328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:02:03.111193 10328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:02:03.111357 10328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:02:03.111646 10328 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:02:03.111999 10328 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:02:03.112033 10328 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:02:03.112077 10328 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:02:03.112104 10328 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:02:03.117878 10328 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.195:46867
I20260929 11:02:03.117974 10456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.195:46867 every 8 connection(s)
I20260929 11:02:03.118310 10328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
I20260929 11:02:03.121876 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 10328
I20260929 11:02:03.122013 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance
I20260929 11:02:03.122246 27107 external_mini_cluster.cc:1428] Setting key b3c53da07fca295d49c86dbf7dfd6748
Sep 29 11:02:03 dist-test-slave-2lzq krb5kdc[9953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679723, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:02:03.128890 10056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:03.119077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:43479 (local address 127.26.120.254:44319)
0929 11:02:03.119161 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0929 11:02:03.119163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:02:03.119321 (+ 158us) server_negotiation.cc:407] Connection header received
0929 11:02:03.119854 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:02:03.119856 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:02:03.119867 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:02:03.119942 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:02:03.120915 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:03.121433 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:03.122838 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:03.123104 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:03.123199 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:03.125369 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:02:03.125371 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:02:03.125371 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:02:03.125379 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:02:03.127198 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:02:03.127751 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:02:03.127752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:02:03.127753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:02:03.127822 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:02:03.128138 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:02:03.128139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:02:03.128140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:02:03.128278 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:02:03.128364 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 11:02:03.128683 (+ 319us) server_negotiation.cc:299] Negotiation successful
0929 11:02:03.128779 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 11:02:03.129117 10459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:03.119038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.195:43479)
0929 11:02:03.119195 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:03.119232 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 11:02:03.119745 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:02:03.120013 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:02:03.120017 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:02:03.120598 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:02:03.120808 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:03.120813 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:03.121536 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:03.121537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:03.122696 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:03.122699 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:03.123211 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:03.123212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:03.123346 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:03.123878 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:02:03.123886 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:02:03.125268 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:02:03.127336 (+ 2068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:02:03.127344 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:02:03.127347 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:02:03.127657 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:02:03.127936 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:02:03.127938 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:02:03.127938 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:02:03.128061 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:02:03.128362 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:02:03.128371 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:02:03.128522 (+ 151us) client_negotiation.cc:769] Sending connection context
0929 11:02:03.128642 (+ 120us) client_negotiation.cc:240] Negotiation successful
0929 11:02:03.128822 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":96,"threads_started":1}
I20260929 11:02:03.129818 10457 heartbeater.cc:347] Connected to a master server at 127.26.120.254:44319
I20260929 11:02:03.129915 10457 heartbeater.cc:464] Registering TS with master...
I20260929 11:02:03.130187 10457 heartbeater.cc:511] Master 127.26.120.254:44319 requested a full tablet report, sending...
I20260929 11:02:03.130772 9998 ts_manager.cc:194] Registered new tserver with Master: 46f46a759dfc42838abffdd16b09d2d7 (127.26.120.195:46867)
I20260929 11:02:03.131181 9998 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.195@KRBTEST.COM'} at 127.26.120.195:43479
I20260929 11:02:03.134338 27107 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 11:02:03.141577 10056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:03.135616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46152 (local address 127.26.120.254:44319)
0929 11:02:03.135698 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0929 11:02:03.135700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:02:03.135797 (+ 97us) server_negotiation.cc:407] Connection header received
0929 11:02:03.135886 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:02:03.135887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:02:03.135897 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:02:03.135975 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:02:03.136538 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:03.137029 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:03.137771 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:03.137994 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:03.138088 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:03.139245 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:02:03.139247 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:02:03.139247 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:02:03.139254 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:02:03.140404 (+ 1150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:02:03.140761 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:02:03.140762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:02:03.140763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:02:03.140823 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:02:03.141021 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:02:03.141022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:02:03.141022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:02:03.141173 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:02:03.141283 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0929 11:02:03.141395 (+ 112us) server_negotiation.cc:299] Negotiation successful
0929 11:02:03.141472 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 11:02:03.143153 9998 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46152:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 11:02:03.144735 9998 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 11:02:03.151453 10472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:03.147362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34672 (local address 127.26.120.194:35161)
0929 11:02:03.147554 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0929 11:02:03.147557 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:02:03.147567 (+ 10us) server_negotiation.cc:407] Connection header received
0929 11:02:03.147601 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:02:03.147603 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:02:03.147620 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:02:03.148000 (+ 380us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:02:03.148075 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:03.149088 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:03.150230 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:03.150982 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:03.151056 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:03.151115 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 11:02:03.151321 (+ 206us) server_negotiation.cc:299] Negotiation successful
0929 11:02:03.151364 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":94,"threads_started":1}
I20260929 11:02:03.151525 10469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:03.147066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:44527 (local address 127.0.0.1:46954)
0929 11:02:03.147228 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:03.147271 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 11:02:03.147336 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:02:03.147706 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:02:03.147710 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:02:03.147712 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:02:03.147790 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:03.147792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:03.149081 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:03.149082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:03.150225 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:03.150227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:03.151092 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:03.151094 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:03.151201 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:03.151211 (+ 10us) client_negotiation.cc:769] Sending connection context
0929 11:02:03.151298 (+ 87us) client_negotiation.cc:240] Negotiation successful
0929 11:02:03.151360 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":12,"thread_start_us":57,"threads_started":1}
I20260929 11:02:03.151533 10468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:03.147097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:35161 (local address 127.0.0.1:34672)
0929 11:02:03.147228 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:03.147271 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 11:02:03.147336 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:02:03.147718 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:02:03.147719 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:02:03.147721 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:02:03.147792 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:03.147794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:03.149163 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:03.149164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:03.150127 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:03.150130 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:03.151094 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:03.151095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:03.151195 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:03.151211 (+ 16us) client_negotiation.cc:769] Sending connection context
0929 11:02:03.151298 (+ 87us) client_negotiation.cc:240] Negotiation successful
0929 11:02:03.151370 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":2,"thread_start_us":77,"threads_started":1}
I20260929 11:02:03.152186 10470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:03.147168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46954 (local address 127.26.120.193:44527)
0929 11:02:03.147520 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0929 11:02:03.147526 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 11:02:03.147536 (+ 10us) server_negotiation.cc:407] Connection header received
0929 11:02:03.147579 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:02:03.147581 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:02:03.147597 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:02:03.147682 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:02:03.147854 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:03.148984 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:03.150298 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:03.150974 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:03.151057 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:03.151115 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 11:02:03.151954 (+ 839us) server_negotiation.cc:299] Negotiation successful
0929 11:02:03.152011 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":75,"threads_started":1}
I20260929 11:02:03.152422 10257 tablet_service.cc:1467] Processing CreateTablet for tablet 6d18edbb4cb0467baa4c1f90cbc87d15 (DEFAULT_TABLE table=test-table [id=21fd3d59c9d5413b863b228495e3b186]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:02:03.152812 10257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d18edbb4cb0467baa4c1f90cbc87d15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:02:03.152940 10123 tablet_service.cc:1467] Processing CreateTablet for tablet 6d18edbb4cb0467baa4c1f90cbc87d15 (DEFAULT_TABLE table=test-table [id=21fd3d59c9d5413b863b228495e3b186]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:02:03.153230 10123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d18edbb4cb0467baa4c1f90cbc87d15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:02:03.155828 10474 tablet_bootstrap.cc:492] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 0d9c8dbb6e1e49a59e61c94c6c79e074: Bootstrap starting.
I20260929 11:02:03.155891 10475 tablet_bootstrap.cc:492] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 20585d017b4b4bfb85a733ae593ebbeb: Bootstrap starting.
I20260929 11:02:03.156387 10471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:03.147261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:46867 (local address 127.0.0.1:35852)
0929 11:02:03.147621 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:03.147630 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 11:02:03.147714 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:02:03.147948 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:02:03.147948 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:02:03.147950 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:02:03.147985 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:03.147986 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:03.148882 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:03.148883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:03.150224 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:03.150227 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:03.156085 (+ 5858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:03.156087 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:03.156163 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:03.156170 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 11:02:03.156242 (+ 72us) client_negotiation.cc:240] Negotiation successful
0929 11:02:03.156303 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":3,"thread_start_us":65,"threads_started":1}
I20260929 11:02:03.156440 10473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:03.147432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35852 (local address 127.26.120.195:46867)
0929 11:02:03.147574 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0929 11:02:03.147579 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 11:02:03.147772 (+ 193us) server_negotiation.cc:407] Connection header received
0929 11:02:03.147814 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:02:03.147817 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:02:03.147835 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:02:03.147911 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:02:03.148065 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:03.148768 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:03.151156 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:03.152516 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:03.152610 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:03.152666 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 11:02:03.156287 (+ 3621us) server_negotiation.cc:299] Negotiation successful
0929 11:02:03.156356 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":67,"threads_started":1}
I20260929 11:02:03.156584 10474 tablet_bootstrap.cc:654] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 0d9c8dbb6e1e49a59e61c94c6c79e074: Neither blocks nor log segments found. Creating new log.
I20260929 11:02:03.156625 10475 tablet_bootstrap.cc:654] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 20585d017b4b4bfb85a733ae593ebbeb: Neither blocks nor log segments found. Creating new log.
I20260929 11:02:03.157163 10474 log.cc:824] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 0d9c8dbb6e1e49a59e61c94c6c79e074: Log is configured to *not* fsync() on all Append() calls
I20260929 11:02:03.157171 10475 log.cc:824] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 20585d017b4b4bfb85a733ae593ebbeb: Log is configured to *not* fsync() on all Append() calls
I20260929 11:02:03.157472 10391 tablet_service.cc:1467] Processing CreateTablet for tablet 6d18edbb4cb0467baa4c1f90cbc87d15 (DEFAULT_TABLE table=test-table [id=21fd3d59c9d5413b863b228495e3b186]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:02:03.157796 10391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d18edbb4cb0467baa4c1f90cbc87d15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:02:03.157897 10475 tablet_bootstrap.cc:492] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 20585d017b4b4bfb85a733ae593ebbeb: No bootstrap required, opened a new log
I20260929 11:02:03.157927 10474 tablet_bootstrap.cc:492] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 0d9c8dbb6e1e49a59e61c94c6c79e074: No bootstrap required, opened a new log
I20260929 11:02:03.157943 10475 ts_tablet_manager.cc:1397] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 20585d017b4b4bfb85a733ae593ebbeb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 11:02:03.157980 10474 ts_tablet_manager.cc:1397] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 0d9c8dbb6e1e49a59e61c94c6c79e074: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 11:02:03.159539 10475 raft_consensus.cc:348] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 20585d017b4b4bfb85a733ae593ebbeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20585d017b4b4bfb85a733ae593ebbeb" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44527 } } peers { permanent_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35161 } } peers { permanent_uuid: "46f46a759dfc42838abffdd16b09d2d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46867 } }
I20260929 11:02:03.159672 10475 raft_consensus.cc:374] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 20585d017b4b4bfb85a733ae593ebbeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:02:03.159693 10475 raft_consensus.cc:729] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 20585d017b4b4bfb85a733ae593ebbeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20585d017b4b4bfb85a733ae593ebbeb, State: Initialized, Role: FOLLOWER
I20260929 11:02:03.159819 10475 consensus_queue.cc:260] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 20585d017b4b4bfb85a733ae593ebbeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20585d017b4b4bfb85a733ae593ebbeb" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44527 } } peers { permanent_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35161 } } peers { permanent_uuid: "46f46a759dfc42838abffdd16b09d2d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46867 } }
I20260929 11:02:03.160094 10475 ts_tablet_manager.cc:1428] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 20585d017b4b4bfb85a733ae593ebbeb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 11:02:03.160095 10189 heartbeater.cc:503] Master 127.26.120.254:44319 was elected leader, sending a full tablet report...
I20260929 11:02:03.160144 10474 raft_consensus.cc:348] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 0d9c8dbb6e1e49a59e61c94c6c79e074 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20585d017b4b4bfb85a733ae593ebbeb" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44527 } } peers { permanent_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35161 } } peers { permanent_uuid: "46f46a759dfc42838abffdd16b09d2d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46867 } }
I20260929 11:02:03.160293 10474 raft_consensus.cc:374] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 0d9c8dbb6e1e49a59e61c94c6c79e074 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:02:03.160327 10474 raft_consensus.cc:729] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 0d9c8dbb6e1e49a59e61c94c6c79e074 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9c8dbb6e1e49a59e61c94c6c79e074, State: Initialized, Role: FOLLOWER
I20260929 11:02:03.160441 10474 consensus_queue.cc:260] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 0d9c8dbb6e1e49a59e61c94c6c79e074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20585d017b4b4bfb85a733ae593ebbeb" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44527 } } peers { permanent_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35161 } } peers { permanent_uuid: "46f46a759dfc42838abffdd16b09d2d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46867 } }
I20260929 11:02:03.160724 10474 ts_tablet_manager.cc:1428] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 0d9c8dbb6e1e49a59e61c94c6c79e074: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 11:02:03.160738 10323 heartbeater.cc:503] Master 127.26.120.254:44319 was elected leader, sending a full tablet report...
I20260929 11:02:03.161351 10480 tablet_bootstrap.cc:492] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 46f46a759dfc42838abffdd16b09d2d7: Bootstrap starting.
I20260929 11:02:03.162099 10480 tablet_bootstrap.cc:654] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 46f46a759dfc42838abffdd16b09d2d7: Neither blocks nor log segments found. Creating new log.
I20260929 11:02:03.162591 10480 log.cc:824] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 46f46a759dfc42838abffdd16b09d2d7: Log is configured to *not* fsync() on all Append() calls
I20260929 11:02:03.163307 10480 tablet_bootstrap.cc:492] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 46f46a759dfc42838abffdd16b09d2d7: No bootstrap required, opened a new log
I20260929 11:02:03.163359 10480 ts_tablet_manager.cc:1397] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 46f46a759dfc42838abffdd16b09d2d7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 11:02:03.164656 10480 raft_consensus.cc:348] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 46f46a759dfc42838abffdd16b09d2d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20585d017b4b4bfb85a733ae593ebbeb" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44527 } } peers { permanent_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35161 } } peers { permanent_uuid: "46f46a759dfc42838abffdd16b09d2d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46867 } }
I20260929 11:02:03.164784 10480 raft_consensus.cc:374] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 46f46a759dfc42838abffdd16b09d2d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:02:03.164806 10480 raft_consensus.cc:729] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 46f46a759dfc42838abffdd16b09d2d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46f46a759dfc42838abffdd16b09d2d7, State: Initialized, Role: FOLLOWER
I20260929 11:02:03.164942 10480 consensus_queue.cc:260] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 46f46a759dfc42838abffdd16b09d2d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20585d017b4b4bfb85a733ae593ebbeb" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44527 } } peers { permanent_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35161 } } peers { permanent_uuid: "46f46a759dfc42838abffdd16b09d2d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46867 } }
I20260929 11:02:03.165192 10480 ts_tablet_manager.cc:1428] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 46f46a759dfc42838abffdd16b09d2d7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 11:02:03.165237 10457 heartbeater.cc:503] Master 127.26.120.254:44319 was elected leader, sending a full tablet report...
W20260929 11:02:03.195297 10324 tablet.cc:2369] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 0d9c8dbb6e1e49a59e61c94c6c79e074: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 11:02:03.236039 10479 raft_consensus.cc:482] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 0d9c8dbb6e1e49a59e61c94c6c79e074 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 11:02:03.236171 10479 raft_consensus.cc:504] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 0d9c8dbb6e1e49a59e61c94c6c79e074 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20585d017b4b4bfb85a733ae593ebbeb" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44527 } } peers { permanent_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35161 } } peers { permanent_uuid: "46f46a759dfc42838abffdd16b09d2d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46867 } }
I20260929 11:02:03.236611 10479 leader_election.cc:288] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 0d9c8dbb6e1e49a59e61c94c6c79e074 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20585d017b4b4bfb85a733ae593ebbeb (127.26.120.193:44527), 46f46a759dfc42838abffdd16b09d2d7 (127.26.120.195:46867)
I20260929 11:02:03.239827 10325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:03.236785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:44527 (local address 127.26.120.194:37255)
0929 11:02:03.236978 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:03.236991 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 11:02:03.237050 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:02:03.237240 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:02:03.237242 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:02:03.237244 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:02:03.237334 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:03.237337 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:03.237987 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:03.237988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:03.239001 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:03.239004 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:03.239555 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:03.239556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:03.239638 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:03.239641 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 11:02:03.239711 (+ 70us) client_negotiation.cc:240] Negotiation successful
0929 11:02:03.239753 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144}
I20260929 11:02:03.239827 10470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:03.236901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:37255 (local address 127.26.120.193:44527)
0929 11:02:03.236966 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 11:02:03.236968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:02:03.237054 (+ 86us) server_negotiation.cc:407] Connection header received
0929 11:02:03.237135 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:02:03.237136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:02:03.237146 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:02:03.237217 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:02:03.237410 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:03.237893 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:03.239092 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:03.239471 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:03.239558 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:03.239568 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0929 11:02:03.239729 (+ 161us) server_negotiation.cc:299] Negotiation successful
0929 11:02:03.239765 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 11:02:03.239938 10483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:03.237019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:46867 (local address 127.26.120.194:49797)
0929 11:02:03.237232 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:03.237246 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 11:02:03.237326 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:02:03.237492 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:02:03.237493 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:02:03.237495 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:02:03.237571 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:03.237573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:03.238263 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:03.238264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:03.239100 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:03.239102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:03.239675 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:03.239676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:03.239753 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:03.239757 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 11:02:03.239816 (+ 59us) client_negotiation.cc:240] Negotiation successful
0929 11:02:03.239855 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":123,"threads_started":1}
I20260929 11:02:03.239928 10473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:03.237128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:49797 (local address 127.26.120.195:46867)
0929 11:02:03.237181 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0929 11:02:03.237183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:02:03.237321 (+ 138us) server_negotiation.cc:407] Connection header received
0929 11:02:03.237383 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:02:03.237384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:02:03.237394 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:02:03.237467 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:02:03.237646 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:03.238177 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:03.239195 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:03.239578 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:03.239661 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:03.239669 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 11:02:03.239833 (+ 164us) server_negotiation.cc:299] Negotiation successful
0929 11:02:03.239869 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 11:02:03.240157 10143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d18edbb4cb0467baa4c1f90cbc87d15" candidate_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20585d017b4b4bfb85a733ae593ebbeb" is_pre_election: true
I20260929 11:02:03.240351 10143 raft_consensus.cc:2454] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 20585d017b4b4bfb85a733ae593ebbeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d9c8dbb6e1e49a59e61c94c6c79e074 in term 0.
I20260929 11:02:03.240492 10411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d18edbb4cb0467baa4c1f90cbc87d15" candidate_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46f46a759dfc42838abffdd16b09d2d7" is_pre_election: true
I20260929 11:02:03.240609 10210 leader_election.cc:302] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 0d9c8dbb6e1e49a59e61c94c6c79e074 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9c8dbb6e1e49a59e61c94c6c79e074, 20585d017b4b4bfb85a733ae593ebbeb; no voters:
I20260929 11:02:03.240672 10411 raft_consensus.cc:2454] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 46f46a759dfc42838abffdd16b09d2d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d9c8dbb6e1e49a59e61c94c6c79e074 in term 0.
I20260929 11:02:03.240742 10479 raft_consensus.cc:2781] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 0d9c8dbb6e1e49a59e61c94c6c79e074 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 11:02:03.240788 10479 raft_consensus.cc:482] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 0d9c8dbb6e1e49a59e61c94c6c79e074 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 11:02:03.240841 10479 raft_consensus.cc:3037] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 0d9c8dbb6e1e49a59e61c94c6c79e074 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:02:03.241539 10479 raft_consensus.cc:504] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 0d9c8dbb6e1e49a59e61c94c6c79e074 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20585d017b4b4bfb85a733ae593ebbeb" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44527 } } peers { permanent_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35161 } } peers { permanent_uuid: "46f46a759dfc42838abffdd16b09d2d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46867 } }
I20260929 11:02:03.241667 10479 leader_election.cc:288] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 0d9c8dbb6e1e49a59e61c94c6c79e074 [CANDIDATE]: Term 1 election: Requested vote from peers 20585d017b4b4bfb85a733ae593ebbeb (127.26.120.193:44527), 46f46a759dfc42838abffdd16b09d2d7 (127.26.120.195:46867)
I20260929 11:02:03.241840 10143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d18edbb4cb0467baa4c1f90cbc87d15" candidate_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20585d017b4b4bfb85a733ae593ebbeb"
I20260929 11:02:03.241858 10411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d18edbb4cb0467baa4c1f90cbc87d15" candidate_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46f46a759dfc42838abffdd16b09d2d7"
I20260929 11:02:03.241909 10143 raft_consensus.cc:3037] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 20585d017b4b4bfb85a733ae593ebbeb [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:02:03.241950 10411 raft_consensus.cc:3037] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 46f46a759dfc42838abffdd16b09d2d7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:02:03.242651 10143 raft_consensus.cc:2454] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 20585d017b4b4bfb85a733ae593ebbeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d9c8dbb6e1e49a59e61c94c6c79e074 in term 1.
I20260929 11:02:03.242676 10411 raft_consensus.cc:2454] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 46f46a759dfc42838abffdd16b09d2d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d9c8dbb6e1e49a59e61c94c6c79e074 in term 1.
I20260929 11:02:03.242820 10210 leader_election.cc:302] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 0d9c8dbb6e1e49a59e61c94c6c79e074 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9c8dbb6e1e49a59e61c94c6c79e074, 46f46a759dfc42838abffdd16b09d2d7; no voters:
I20260929 11:02:03.242937 10479 raft_consensus.cc:2781] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 0d9c8dbb6e1e49a59e61c94c6c79e074 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:02:03.243129 10479 raft_consensus.cc:686] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 0d9c8dbb6e1e49a59e61c94c6c79e074 [term 1 LEADER]: Becoming Leader. State: Replica: 0d9c8dbb6e1e49a59e61c94c6c79e074, State: Running, Role: LEADER
I20260929 11:02:03.243264 10479 consensus_queue.cc:237] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 0d9c8dbb6e1e49a59e61c94c6c79e074 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20585d017b4b4bfb85a733ae593ebbeb" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44527 } } peers { permanent_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35161 } } peers { permanent_uuid: "46f46a759dfc42838abffdd16b09d2d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46867 } }
I20260929 11:02:03.244084 9998 catalog_manager.cc:5495] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 0d9c8dbb6e1e49a59e61c94c6c79e074 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d9c8dbb6e1e49a59e61c94c6c79e074 (127.26.120.194). New cstate: current_term: 1 leader_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20585d017b4b4bfb85a733ae593ebbeb" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46f46a759dfc42838abffdd16b09d2d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46867 } health_report { overall_health: UNKNOWN } } }
I20260929 11:02:03.265909 10472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:03.263631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34686 (local address 127.26.120.194:35161)
0929 11:02:03.263689 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 11:02:03.263692 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:02:03.263700 (+ 8us) server_negotiation.cc:407] Connection header received
0929 11:02:03.263744 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:02:03.263745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:02:03.263761 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:02:03.263823 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 11:02:03.263931 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:03.264463 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:03.265202 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:03.265419 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:03.265488 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:03.265614 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 11:02:03.265711 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 11:02:03.265791 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 11:02:03.265812 (+ 21us) server_negotiation.cc:299] Negotiation successful
0929 11:02:03.265830 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260929 11:02:03.267992 10411 raft_consensus.cc:1261] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 46f46a759dfc42838abffdd16b09d2d7 [term 1 FOLLOWER]: Refusing update from remote peer 0d9c8dbb6e1e49a59e61c94c6c79e074: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 11:02:03.268066 10143 raft_consensus.cc:1261] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 20585d017b4b4bfb85a733ae593ebbeb [term 1 FOLLOWER]: Refusing update from remote peer 0d9c8dbb6e1e49a59e61c94c6c79e074: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 11:02:03.268411 10479 consensus_queue.cc:1035] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 0d9c8dbb6e1e49a59e61c94c6c79e074 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20585d017b4b4bfb85a733ae593ebbeb" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 11:02:03.268558 10484 consensus_queue.cc:1035] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 0d9c8dbb6e1e49a59e61c94c6c79e074 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46f46a759dfc42838abffdd16b09d2d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 11:02:03.271147 10488 mvcc.cc:205] Tried to move back new op lower bound from 7334624146502905856 to 7334624146404995072. Current Snapshot: MvccSnapshot[applied={T|T < 7334624146502905856}]
I20260929 11:02:03.271551 10489 mvcc.cc:205] Tried to move back new op lower bound from 7334624146502905856 to 7334624146404995072. Current Snapshot: MvccSnapshot[applied={T|T < 7334624146502905856}]
I20260929 11:02:03.273202 9998 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46152:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 11:02:03.273304 9998 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46152:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 11:02:03.274091 9998 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4: Sending DeleteTablet for 3 replicas of tablet 6d18edbb4cb0467baa4c1f90cbc87d15
I20260929 11:02:03.274412 10123 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d18edbb4cb0467baa4c1f90cbc87d15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:02:03 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:46954
I20260929 11:02:03.274422 10257 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d18edbb4cb0467baa4c1f90cbc87d15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:02:03 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:34672
I20260929 11:02:03.274505 10391 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d18edbb4cb0467baa4c1f90cbc87d15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:02:03 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:35852
I20260929 11:02:03.274645 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 10060
I20260929 11:02:03.274670 10497 tablet_replica.cc:330] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 0d9c8dbb6e1e49a59e61c94c6c79e074: stopping tablet replica
I20260929 11:02:03.274724 10499 tablet_replica.cc:330] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 46f46a759dfc42838abffdd16b09d2d7: stopping tablet replica
I20260929 11:02:03.274785 10497 raft_consensus.cc:2229] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 0d9c8dbb6e1e49a59e61c94c6c79e074 [term 1 LEADER]: Raft consensus shutting down.
I20260929 11:02:03.274874 10499 raft_consensus.cc:2229] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 46f46a759dfc42838abffdd16b09d2d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 11:02:03.274979 10497 raft_consensus.cc:2258] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 0d9c8dbb6e1e49a59e61c94c6c79e074 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 11:02:03.275032 10499 raft_consensus.cc:2258] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 46f46a759dfc42838abffdd16b09d2d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 11:02:03.275496 10497 ts_tablet_manager.cc:1905] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 0d9c8dbb6e1e49a59e61c94c6c79e074: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 11:02:03.275497 10499 ts_tablet_manager.cc:1905] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 46f46a759dfc42838abffdd16b09d2d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 11:02:03.277297 10497 ts_tablet_manager.cc:1918] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 0d9c8dbb6e1e49a59e61c94c6c79e074: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 11:02:03.277446 10497 log.cc:1196] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 0d9c8dbb6e1e49a59e61c94c6c79e074: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/wal/wals/6d18edbb4cb0467baa4c1f90cbc87d15
I20260929 11:02:03.277493 10499 ts_tablet_manager.cc:1918] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 46f46a759dfc42838abffdd16b09d2d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 11:02:03.277637 10499 log.cc:1196] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 46f46a759dfc42838abffdd16b09d2d7: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/wal/wals/6d18edbb4cb0467baa4c1f90cbc87d15
I20260929 11:02:03.277787 10497 ts_tablet_manager.cc:1939] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 0d9c8dbb6e1e49a59e61c94c6c79e074: Deleting consensus metadata
I20260929 11:02:03.278016 10499 ts_tablet_manager.cc:1939] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 46f46a759dfc42838abffdd16b09d2d7: Deleting consensus metadata
I20260929 11:02:03.278321 9985 catalog_manager.cc:4841] TS 0d9c8dbb6e1e49a59e61c94c6c79e074 (127.26.120.194:35161): tablet 6d18edbb4cb0467baa4c1f90cbc87d15 (table test-table [id=21fd3d59c9d5413b863b228495e3b186]) successfully deleted
I20260929 11:02:03.278535 9982 catalog_manager.cc:4841] TS 46f46a759dfc42838abffdd16b09d2d7 (127.26.120.195:46867): tablet 6d18edbb4cb0467baa4c1f90cbc87d15 (table test-table [id=21fd3d59c9d5413b863b228495e3b186]) successfully deleted
W20260929 11:02:03.287796 9982 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.120.193:44527 (error 108)
I20260929 11:02:03.288385 10471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:03.288240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:44527 (local address 127.0.0.1:46968)
0929 11:02:03.288306 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:03.288346 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.193:44527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260929 11:02:03.288421 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 10194
W20260929 11:02:03.290228 9982 catalog_manager.cc:4584] TS 20585d017b4b4bfb85a733ae593ebbeb (127.26.120.193:44527): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6d18edbb4cb0467baa4c1f90cbc87d15: Network error: Client connection negotiation failed: client connection to 127.26.120.193:44527: connect: Connection refused (error 111)
I20260929 11:02:03.293490 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 10328
I20260929 11:02:03.297931 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 9966
I20260929 11:02:23.303524 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:44319
--webserver_interface=127.26.120.254
--webserver_port=36475
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:34649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:44319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:02:23.384999 10501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:02:23.385213 10501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:02:23.385236 10501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:02:23.386495 10501 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 11:02:23.386539 10501 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 11:02:23.386554 10501 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 11:02:23.386564 10501 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:02:23.386579 10501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:02:23.386591 10501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:02:23.386634 10501 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:02:23.386651 10501 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:02:23.388238 10501 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:34649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:44319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:44319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=36475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:02:23.388494 10501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:02:23.388751 10501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:02:23.397845 10510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:02:23.397948 10507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:02:23.397845 10508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:02:23.397981 10501 server_base.cc:1034] running on GCE node
I20260929 11:02:23.398590 10501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:02:23.398939 10501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:02:23.400084 10501 hybrid_clock.cc:648] HybridClock initialized: now 1790679743400063 us; error 37 us; skew 500 ppm
Sep 29 11:02:23 dist-test-slave-2lzq krb5kdc[9953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679743, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:02:23.407721 10501 init.cc:377] Logged in from keytab as kudu/127.26.120.254@KRBTEST.COM (short username kudu)
I20260929 11:02:23.408238 10501 webserver.cc:460] Webserver started at http://127.26.120.254:36475/ using document root <none> and password file <none>
I20260929 11:02:23.408497 10501 fs_manager.cc:359] Metadata directory not provided
I20260929 11:02:23.408581 10501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:02:23.410007 10501 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 11:02:23.410688 10517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:02:23.410861 10501 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 11:02:23.410982 10501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "037fbc5362fc4c5ab9f7ece66813ebe4"
format_stamp: "Formatted at 2026-09-29 11:02:02 on dist-test-slave-2lzq"
server_key: "307b583f309db733d33df0a955b7cbc0"
server_key_iv: "e6a04accb7f7dddea1607b03bbd44190"
server_key_version: "encryptionkey@0"
I20260929 11:02:23.411393 10501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:02:23.429859 10501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:02:23.431900 10501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:02:23.432080 10501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:02:23.436472 10501 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:44319
I20260929 11:02:23.436549 10569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:44319 every 8 connection(s)
I20260929 11:02:23.436954 10501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:02:23.438632 10570 sys_catalog.cc:263] Verifying existing consensus state
I20260929 11:02:23.438860 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 10501
I20260929 11:02:23.439144 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.193:44527
--local_ip_for_outbound_sockets=127.26.120.193
--tserver_master_addrs=127.26.120.254:44319
--webserver_port=44249
--webserver_interface=127.26.120.193
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:34649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 11:02:23.439406 10570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4: Bootstrap starting.
I20260929 11:02:23.448168 10570 log.cc:824] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4: Log is configured to *not* fsync() on all Append() calls
I20260929 11:02:23.450312 10570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 11:02:23.450619 10570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4: Bootstrap complete.
I20260929 11:02:23.453028 10570 raft_consensus.cc:348] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "037fbc5362fc4c5ab9f7ece66813ebe4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44319 } }
I20260929 11:02:23.453250 10570 raft_consensus.cc:729] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 037fbc5362fc4c5ab9f7ece66813ebe4, State: Initialized, Role: FOLLOWER
I20260929 11:02:23.453433 10570 consensus_queue.cc:260] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "037fbc5362fc4c5ab9f7ece66813ebe4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44319 } }
I20260929 11:02:23.453493 10570 raft_consensus.cc:388] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 11:02:23.453562 10570 raft_consensus.cc:482] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 11:02:23.453613 10570 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [term 1 FOLLOWER]: Advancing to term 2
I20260929 11:02:23.454713 10570 raft_consensus.cc:504] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "037fbc5362fc4c5ab9f7ece66813ebe4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44319 } }
I20260929 11:02:23.454826 10570 leader_election.cc:302] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 037fbc5362fc4c5ab9f7ece66813ebe4; no voters:
I20260929 11:02:23.455040 10570 leader_election.cc:288] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 11:02:23.455077 10574 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 11:02:23.455248 10574 raft_consensus.cc:686] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [term 2 LEADER]: Becoming Leader. State: Replica: 037fbc5362fc4c5ab9f7ece66813ebe4, State: Running, Role: LEADER
I20260929 11:02:23.455323 10570 sys_catalog.cc:564] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:02:23.455391 10574 consensus_queue.cc:237] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "037fbc5362fc4c5ab9f7ece66813ebe4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44319 } }
I20260929 11:02:23.455650 10574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 037fbc5362fc4c5ab9f7ece66813ebe4. Latest consensus state: current_term: 2 leader_uuid: "037fbc5362fc4c5ab9f7ece66813ebe4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "037fbc5362fc4c5ab9f7ece66813ebe4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44319 } } }
I20260929 11:02:23.455736 10574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [sys.catalog]: This master's current role is: LEADER
I20260929 11:02:23.455657 10575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "037fbc5362fc4c5ab9f7ece66813ebe4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "037fbc5362fc4c5ab9f7ece66813ebe4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44319 } } }
I20260929 11:02:23.455941 10575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [sys.catalog]: This master's current role is: LEADER
I20260929 11:02:23.456322 10583 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:02:23.457135 10583 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:02:23.457294 10583 catalog_manager.cc:1259] Loaded cluster ID: 61e7e82bb0c448e3b54e7fcdfde61c94
I20260929 11:02:23.457373 10583 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 11:02:23.458770 10583 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:02:23.458936 10583 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4: Loaded TSK: 0
I20260929 11:02:23.468170 10583 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4: Generated new TSK 1
I20260929 11:02:23.468257 10583 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 11:02:23.523061 10572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:02:23.523257 10572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:02:23.523279 10572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:02:23.524441 10572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:02:23.524482 10572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:02:23.524508 10572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.193
I20260929 11:02:23.525949 10572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:34649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.193:44527
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.120.193
--webserver_port=44249
--tserver_master_addrs=127.26.120.254:44319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.193
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:02:23.526198 10572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:02:23.526451 10572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:02:23.532217 10597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:02:23.532223 10598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:02:23.532387 10600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:02:23.532521 10572 server_base.cc:1034] running on GCE node
I20260929 11:02:23.532730 10572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:02:23.533005 10572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:02:23.534116 10572 hybrid_clock.cc:648] HybridClock initialized: now 1790679743534102 us; error 27 us; skew 500 ppm
Sep 29 11:02:23 dist-test-slave-2lzq krb5kdc[9953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679743, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:02:23.536751 10572 init.cc:377] Logged in from keytab as kudu/127.26.120.193@KRBTEST.COM (short username kudu)
I20260929 11:02:23.537343 10572 webserver.cc:460] Webserver started at http://127.26.120.193:44249/ using document root <none> and password file <none>
I20260929 11:02:23.537627 10572 fs_manager.cc:359] Metadata directory not provided
I20260929 11:02:23.537712 10572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:02:23.539640 10572 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:02:23.540500 10607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:02:23.540762 10572 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:02:23.540874 10572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/wal
uuid: "20585d017b4b4bfb85a733ae593ebbeb"
format_stamp: "Formatted at 2026-09-29 11:02:02 on dist-test-slave-2lzq"
server_key: "07c5bcb871d78c26fe0da942254088ac"
server_key_iv: "fb426bbcd89a9cdb68356ee7b1ba8e0c"
server_key_version: "encryptionkey@0"
I20260929 11:02:23.541323 10572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:02:23.567427 10572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:02:23.570031 10572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:02:23.570226 10572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:02:23.570505 10572 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:02:23.571012 10614 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260929 11:02:23.572477 10572 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 11:02:23.572542 10572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260929 11:02:23.572579 10572 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 11:02:23.573441 10572 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 11:02:23.573489 10572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260929 11:02:23.573563 10614 tablet_bootstrap.cc:492] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 20585d017b4b4bfb85a733ae593ebbeb: Bootstrap starting.
I20260929 11:02:23.581629 10572 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.193:44527
I20260929 11:02:23.582111 10572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
I20260929 11:02:23.584084 10614 log.cc:824] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 20585d017b4b4bfb85a733ae593ebbeb: Log is configured to *not* fsync() on all Append() calls
I20260929 11:02:23.584523 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 10572
I20260929 11:02:23.584517 10721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.193:44527 every 8 connection(s)
I20260929 11:02:23.584838 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.194:35161
--local_ip_for_outbound_sockets=127.26.120.194
--tserver_master_addrs=127.26.120.254:44319
--webserver_port=40263
--webserver_interface=127.26.120.194
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:34649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 11:02:23.587054 10614 tablet_bootstrap.cc:492] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 20585d017b4b4bfb85a733ae593ebbeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 11:02:23.587456 10614 tablet_bootstrap.cc:492] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 20585d017b4b4bfb85a733ae593ebbeb: Bootstrap complete.
I20260929 11:02:23.588683 10614 ts_tablet_manager.cc:1397] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 20585d017b4b4bfb85a733ae593ebbeb: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.004s
I20260929 11:02:23.590619 10614 raft_consensus.cc:348] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 20585d017b4b4bfb85a733ae593ebbeb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20585d017b4b4bfb85a733ae593ebbeb" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44527 } } peers { permanent_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35161 } } peers { permanent_uuid: "46f46a759dfc42838abffdd16b09d2d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46867 } }
I20260929 11:02:23.591110 10614 raft_consensus.cc:729] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 20585d017b4b4bfb85a733ae593ebbeb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20585d017b4b4bfb85a733ae593ebbeb, State: Initialized, Role: FOLLOWER
I20260929 11:02:23.591322 10614 consensus_queue.cc:260] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 20585d017b4b4bfb85a733ae593ebbeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20585d017b4b4bfb85a733ae593ebbeb" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44527 } } peers { permanent_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35161 } } peers { permanent_uuid: "46f46a759dfc42838abffdd16b09d2d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46867 } }
I20260929 11:02:23.591704 10614 ts_tablet_manager.cc:1428] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 20585d017b4b4bfb85a733ae593ebbeb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
Sep 29 11:02:23 dist-test-slave-2lzq krb5kdc[9953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679743, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:02:23.599280 10724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:23.582904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.193:46295)
0929 11:02:23.585866 (+ 2962us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:23.585918 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 11:02:23.587112 (+ 1194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:02:23.587712 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:02:23.587716 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:02:23.588078 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:02:23.588350 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:23.588357 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:23.590022 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:23.590023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:23.591403 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:23.591407 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:23.592316 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:23.592317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:23.592486 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:23.593129 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:02:23.593141 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:02:23.594996 (+ 1855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:02:23.597400 (+ 2404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:02:23.597407 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:02:23.597409 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:02:23.597715 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:02:23.598019 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:02:23.598020 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:02:23.598020 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:02:23.598186 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:02:23.598646 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:02:23.598653 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:02:23.598806 (+ 153us) client_negotiation.cc:769] Sending connection context
0929 11:02:23.598913 (+ 107us) client_negotiation.cc:240] Negotiation successful
0929 11:02:23.599072 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2920,"thread_start_us":94,"threads_started":1}
I20260929 11:02:23.599619 10725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:23.583030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:46295 (local address 127.26.120.254:44319)
0929 11:02:23.583226 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0929 11:02:23.583237 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 11:02:23.586161 (+ 2924us) server_negotiation.cc:407] Connection header received
0929 11:02:23.587402 (+ 1241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:02:23.587408 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:02:23.587467 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:02:23.587585 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:02:23.588503 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:23.589898 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:23.591516 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:23.592204 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:23.592338 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:23.595129 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:02:23.595134 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:02:23.595134 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:02:23.595147 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:02:23.597313 (+ 2166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:02:23.597824 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:02:23.597828 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:02:23.597828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:02:23.597942 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:02:23.598264 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:02:23.598265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:02:23.598265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:02:23.598564 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:02:23.598644 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 11:02:23.599008 (+ 364us) server_negotiation.cc:299] Negotiation successful
0929 11:02:23.599146 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":100,"threads_started":1}
I20260929 11:02:23.600397 10722 heartbeater.cc:347] Connected to a master server at 127.26.120.254:44319
I20260929 11:02:23.600488 10722 heartbeater.cc:464] Registering TS with master...
I20260929 11:02:23.600654 10722 heartbeater.cc:511] Master 127.26.120.254:44319 requested a full tablet report, sending...
I20260929 11:02:23.601383 10534 ts_manager.cc:194] Registered new tserver with Master: 20585d017b4b4bfb85a733ae593ebbeb (127.26.120.193:44527)
I20260929 11:02:23.602492 10534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.193@KRBTEST.COM'} at 127.26.120.193:46295
I20260929 11:02:23.603456 10722 heartbeater.cc:503] Master 127.26.120.254:44319 was elected leader, sending a full tablet report...
Sep 29 11:02:23 dist-test-slave-2lzq krb5kdc[9953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679743, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for kudu/127.26.120.193@KRBTEST.COM
I20260929 11:02:23.609990 10729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:23.601962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:44527 (local address 127.0.0.1:44598)
0929 11:02:23.602302 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:23.602336 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 11:02:23.602410 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:02:23.602575 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:02:23.602578 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:02:23.602912 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:02:23.602970 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:23.602972 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:23.604030 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:23.604031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:23.604605 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:23.604607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:23.605098 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:23.605099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:23.605180 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:23.605655 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:02:23.605660 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:02:23.606984 (+ 1324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:02:23.608772 (+ 1788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:02:23.608779 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:02:23.608781 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:02:23.608966 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:02:23.609222 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:02:23.609242 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:02:23.609242 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:02:23.609361 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:02:23.609680 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:02:23.609687 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:02:23.609731 (+ 44us) client_negotiation.cc:769] Sending connection context
0929 11:02:23.609810 (+ 79us) client_negotiation.cc:240] Negotiation successful
0929 11:02:23.609918 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":124,"threads_started":1}
I20260929 11:02:23.610042 10730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:23.602078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44598 (local address 127.26.120.193:44527)
0929 11:02:23.602222 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0929 11:02:23.602226 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 11:02:23.602419 (+ 193us) server_negotiation.cc:407] Connection header received
0929 11:02:23.602484 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:02:23.602486 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:02:23.602500 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:02:23.602572 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:02:23.603052 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:23.603956 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:23.604701 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:23.605003 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:23.605094 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:23.607063 (+ 1969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:02:23.607070 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:02:23.607071 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:02:23.607078 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:02:23.608688 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:02:23.609046 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:02:23.609049 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:02:23.609050 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:02:23.609114 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:02:23.609444 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:02:23.609445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:02:23.609445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:02:23.609610 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:02:23.609689 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 11:02:23.609864 (+ 175us) server_negotiation.cc:299] Negotiation successful
0929 11:02:23.609957 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":119,"thread_start_us":73,"threads_started":1}
I20260929 11:02:23.610543 10656 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d18edbb4cb0467baa4c1f90cbc87d15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:02:03 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:44598
I20260929 11:02:23.611121 10731 tablet_replica.cc:330] stopping tablet replica
I20260929 11:02:23.611238 10731 raft_consensus.cc:2229] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 20585d017b4b4bfb85a733ae593ebbeb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 11:02:23.611343 10731 raft_consensus.cc:2258] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 20585d017b4b4bfb85a733ae593ebbeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 11:02:23.611657 10731 ts_tablet_manager.cc:1905] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 20585d017b4b4bfb85a733ae593ebbeb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 11:02:23.613844 10731 ts_tablet_manager.cc:1918] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 20585d017b4b4bfb85a733ae593ebbeb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 11:02:23.613953 10731 log.cc:1196] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 20585d017b4b4bfb85a733ae593ebbeb: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/wal/wals/6d18edbb4cb0467baa4c1f90cbc87d15
I20260929 11:02:23.614234 10731 ts_tablet_manager.cc:1939] T 6d18edbb4cb0467baa4c1f90cbc87d15 P 20585d017b4b4bfb85a733ae593ebbeb: Deleting consensus metadata
I20260929 11:02:23.614775 10518 catalog_manager.cc:4841] TS 20585d017b4b4bfb85a733ae593ebbeb (127.26.120.193:44527): tablet 6d18edbb4cb0467baa4c1f90cbc87d15 (table test-table [id=21fd3d59c9d5413b863b228495e3b186]) successfully deleted
W20260929 11:02:23.671967 10727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:02:23.672163 10727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:02:23.672185 10727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:02:23.673242 10727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:02:23.673275 10727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:02:23.673301 10727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.194
I20260929 11:02:23.674685 10727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:34649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.194:35161
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.120.194
--webserver_port=40263
--tserver_master_addrs=127.26.120.254:44319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.194
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:02:23.674865 10727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:02:23.675040 10727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:02:23.680143 10737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:02:23.680143 10736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:02:23.680213 10739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:02:23.680454 10727 server_base.cc:1034] running on GCE node
I20260929 11:02:23.680646 10727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:02:23.680898 10727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:02:23.682132 10727 hybrid_clock.cc:648] HybridClock initialized: now 1790679743682110 us; error 42 us; skew 500 ppm
Sep 29 11:02:23 dist-test-slave-2lzq krb5kdc[9953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679743, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:02:23.684785 10727 init.cc:377] Logged in from keytab as kudu/127.26.120.194@KRBTEST.COM (short username kudu)
I20260929 11:02:23.685276 10727 webserver.cc:460] Webserver started at http://127.26.120.194:40263/ using document root <none> and password file <none>
I20260929 11:02:23.685535 10727 fs_manager.cc:359] Metadata directory not provided
I20260929 11:02:23.685621 10727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:02:23.687074 10727 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:02:23.687768 10746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:02:23.688000 10727 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:02:23.688114 10727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/wal
uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074"
format_stamp: "Formatted at 2026-09-29 11:02:02 on dist-test-slave-2lzq"
server_key: "f9923899c0f03499f5bb1ff248a5c5e6"
server_key_iv: "28668a3585e22ba56b831a649a0e6206"
server_key_version: "encryptionkey@0"
I20260929 11:02:23.688488 10727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:02:23.699922 10727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:02:23.701889 10727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:02:23.702095 10727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:02:23.702359 10727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:02:23.702673 10727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:02:23.702729 10727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:02:23.702765 10727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:02:23.702791 10727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:02:23.709093 10727 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.194:35161
I20260929 11:02:23.709187 10859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.194:35161 every 8 connection(s)
I20260929 11:02:23.709549 10727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
I20260929 11:02:23.710724 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 10727
I20260929 11:02:23.711021 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.195:46867
--local_ip_for_outbound_sockets=127.26.120.195
--tserver_master_addrs=127.26.120.254:44319
--webserver_port=41745
--webserver_interface=127.26.120.195
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:34649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 11:02:23 dist-test-slave-2lzq krb5kdc[9953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679743, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:02:23.720001 10862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:23.710361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.194:43285)
0929 11:02:23.710542 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:23.710586 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 11:02:23.711165 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:02:23.711432 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:02:23.711436 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:02:23.712041 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:02:23.712251 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:23.712256 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:23.712987 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:23.712988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:23.713819 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:23.713821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:23.714372 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:23.714373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:23.714511 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:23.715135 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:02:23.715145 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:02:23.716628 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:02:23.718623 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:02:23.718629 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:02:23.718631 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:02:23.718855 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:02:23.719082 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:02:23.719082 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:02:23.719083 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:02:23.719179 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:02:23.719485 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:02:23.719489 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:02:23.719612 (+ 123us) client_negotiation.cc:769] Sending connection context
0929 11:02:23.719698 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 11:02:23.719823 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":93,"threads_started":1}
I20260929 11:02:23.719996 10725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:23.710437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:43285 (local address 127.26.120.254:44319)
0929 11:02:23.710499 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0929 11:02:23.710501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:02:23.710684 (+ 183us) server_negotiation.cc:407] Connection header received
0929 11:02:23.711283 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:02:23.711284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:02:23.711300 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:02:23.711356 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:02:23.712376 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:23.712887 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:23.713927 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:23.714282 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:23.714354 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:23.716724 (+ 2370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:02:23.716725 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:02:23.716726 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:02:23.716743 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:02:23.718536 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:02:23.718937 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:02:23.718938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:02:23.718939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:02:23.719010 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:02:23.719247 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:02:23.719248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:02:23.719249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:02:23.719413 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:02:23.719492 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 11:02:23.719760 (+ 268us) server_negotiation.cc:299] Negotiation successful
0929 11:02:23.719868 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 11:02:23.720631 10860 heartbeater.cc:347] Connected to a master server at 127.26.120.254:44319
I20260929 11:02:23.720760 10860 heartbeater.cc:464] Registering TS with master...
I20260929 11:02:23.720985 10860 heartbeater.cc:511] Master 127.26.120.254:44319 requested a full tablet report, sending...
I20260929 11:02:23.721457 10534 ts_manager.cc:194] Registered new tserver with Master: 0d9c8dbb6e1e49a59e61c94c6c79e074 (127.26.120.194:35161)
I20260929 11:02:23.721838 10534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.194@KRBTEST.COM'} at 127.26.120.194:43285
W20260929 11:02:23.797081 10863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:02:23.797287 10863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:02:23.797312 10863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:02:23.798520 10863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:02:23.798563 10863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:02:23.798590 10863 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.195
I20260929 11:02:23.800040 10863 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:34649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.195:46867
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.120.195
--webserver_port=41745
--tserver_master_addrs=127.26.120.254:44319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.195
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:02:23.800275 10863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:02:23.800508 10863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:02:23.805591 10868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:02:23.805593 10871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:02:23.805603 10869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:02:23.805609 10863 server_base.cc:1034] running on GCE node
I20260929 11:02:23.806037 10863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:02:23.806314 10863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:02:23.807462 10863 hybrid_clock.cc:648] HybridClock initialized: now 1790679743807451 us; error 31 us; skew 500 ppm
Sep 29 11:02:23 dist-test-slave-2lzq krb5kdc[9953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679743, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:02:23.810261 10863 init.cc:377] Logged in from keytab as kudu/127.26.120.195@KRBTEST.COM (short username kudu)
I20260929 11:02:23.810849 10863 webserver.cc:460] Webserver started at http://127.26.120.195:41745/ using document root <none> and password file <none>
I20260929 11:02:23.811148 10863 fs_manager.cc:359] Metadata directory not provided
I20260929 11:02:23.811239 10863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:02:23.813349 10863 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 11:02:23.813966 10878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:02:23.814182 10863 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:02:23.814277 10863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/wal
uuid: "46f46a759dfc42838abffdd16b09d2d7"
format_stamp: "Formatted at 2026-09-29 11:02:03 on dist-test-slave-2lzq"
server_key: "99ef178a55e0037763e2479557d74d62"
server_key_iv: "bc24c09cd7a2b69055dcf9ad767d3706"
server_key_version: "encryptionkey@0"
I20260929 11:02:23.814659 10863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:02:23.827778 10863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:02:23.830215 10863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:02:23.830375 10863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:02:23.830621 10863 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:02:23.830940 10863 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:02:23.830994 10863 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:02:23.831039 10863 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:02:23.831063 10863 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:02:23.837150 10863 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.195:46867
I20260929 11:02:23.837191 10991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.195:46867 every 8 connection(s)
I20260929 11:02:23.837574 10863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
Sep 29 11:02:23 dist-test-slave-2lzq krb5kdc[9953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679743, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:02:23.846643 10725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:23.838349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:38481 (local address 127.26.120.254:44319)
0929 11:02:23.838420 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0929 11:02:23.838423 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:02:23.838639 (+ 216us) server_negotiation.cc:407] Connection header received
0929 11:02:23.839187 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:02:23.839189 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:02:23.839202 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:02:23.839255 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:02:23.840223 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:23.840730 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:23.841745 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:23.841952 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:23.842031 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:23.844125 (+ 2094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:02:23.844126 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:02:23.844126 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:02:23.844134 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:02:23.845408 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:02:23.845758 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:02:23.845759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:02:23.845759 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:02:23.845810 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:02:23.846025 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:02:23.846025 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:02:23.846026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:02:23.846196 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:02:23.846250 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 11:02:23.846513 (+ 263us) server_negotiation.cc:299] Negotiation successful
0929 11:02:23.846579 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 11:02:23.846768 10994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:23.838354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.195:38481)
0929 11:02:23.838513 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:23.838551 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 11:02:23.839070 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:02:23.839331 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:02:23.839334 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:02:23.839920 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:02:23.840113 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:23.840117 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:23.840831 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:23.840832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:23.841650 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:23.841652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:23.842050 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:23.842051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:23.842201 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:23.842693 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:02:23.842701 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:02:23.844039 (+ 1338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:02:23.845481 (+ 1442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:02:23.845486 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:02:23.845488 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:02:23.845695 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:02:23.845866 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:02:23.845867 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:02:23.845867 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:02:23.845967 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:02:23.846255 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:02:23.846260 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:02:23.846375 (+ 115us) client_negotiation.cc:769] Sending connection context
0929 11:02:23.846463 (+ 88us) client_negotiation.cc:240] Negotiation successful
0929 11:02:23.846578 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":70,"threads_started":1}
I20260929 11:02:23.847316 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 10863
I20260929 11:02:23.847600 10992 heartbeater.cc:347] Connected to a master server at 127.26.120.254:44319
I20260929 11:02:23.847697 10992 heartbeater.cc:464] Registering TS with master...
I20260929 11:02:23.847865 10992 heartbeater.cc:511] Master 127.26.120.254:44319 requested a full tablet report, sending...
I20260929 11:02:23.848317 10534 ts_manager.cc:194] Registered new tserver with Master: 46f46a759dfc42838abffdd16b09d2d7 (127.26.120.195:46867)
I20260929 11:02:23.848711 10534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.195@KRBTEST.COM'} at 127.26.120.195:38481
I20260929 11:02:23.853674 10725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:23.847681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41626 (local address 127.26.120.254:44319)
0929 11:02:23.847726 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 11:02:23.847728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:02:23.847912 (+ 184us) server_negotiation.cc:407] Connection header received
0929 11:02:23.847969 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:02:23.847969 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:02:23.847980 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:02:23.848038 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:02:23.848711 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:23.849293 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:23.850081 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:23.850290 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:23.850375 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:23.851482 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:02:23.851483 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:02:23.851484 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:02:23.851490 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:02:23.852552 (+ 1062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:02:23.852900 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:02:23.852901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:02:23.852901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:02:23.852962 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:02:23.853161 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:02:23.853162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:02:23.853162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:02:23.853273 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:02:23.853473 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0929 11:02:23.853510 (+ 37us) server_negotiation.cc:299] Negotiation successful
0929 11:02:23.853578 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 11:02:23.854467 27107 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 11:02:23.857179 10725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:23.854807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41642 (local address 127.26.120.254:44319)
0929 11:02:23.854856 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 11:02:23.854858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:02:23.854978 (+ 120us) server_negotiation.cc:407] Connection header received
0929 11:02:23.855027 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:02:23.855028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:02:23.855055 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:02:23.855116 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 11:02:23.855184 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:23.855695 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:23.856595 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:23.856763 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:23.856813 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:23.856952 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 11:02:23.857039 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 11:02:23.857092 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 11:02:23.857123 (+ 31us) server_negotiation.cc:299] Negotiation successful
0929 11:02:23.857136 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260929 11:02:23.857432 10534 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:41642:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 11:02:23.859031 10534 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 11:02:23.861557 10656 tablet_service.cc:1467] Processing CreateTablet for tablet 2499d7a661184028953313273842ba8d (DEFAULT_TABLE table=test-table [id=2585fce8d16a4aba96013b8b7822ebf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:02:23.861790 10656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2499d7a661184028953313273842ba8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:02:23.863319 10614 tablet_bootstrap.cc:492] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb: Bootstrap starting.
I20260929 11:02:23.863696 10614 tablet_bootstrap.cc:654] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb: Neither blocks nor log segments found. Creating new log.
I20260929 11:02:23.864266 10614 tablet_bootstrap.cc:492] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb: No bootstrap required, opened a new log
I20260929 11:02:23.864326 10614 ts_tablet_manager.cc:1397] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 11:02:23.864699 10614 raft_consensus.cc:348] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35161 } } peers { permanent_uuid: "46f46a759dfc42838abffdd16b09d2d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46867 } } peers { permanent_uuid: "20585d017b4b4bfb85a733ae593ebbeb" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44527 } }
I20260929 11:02:23.864800 10614 raft_consensus.cc:374] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:02:23.864856 10614 raft_consensus.cc:729] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20585d017b4b4bfb85a733ae593ebbeb, State: Initialized, Role: FOLLOWER
I20260929 11:02:23.864902 10614 consensus_queue.cc:260] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35161 } } peers { permanent_uuid: "46f46a759dfc42838abffdd16b09d2d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46867 } } peers { permanent_uuid: "20585d017b4b4bfb85a733ae593ebbeb" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44527 } }
I20260929 11:02:23.865007 10614 ts_tablet_manager.cc:1428] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 11:02:23.865617 10997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:23.860932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:46867 (local address 127.0.0.1:55174)
0929 11:02:23.861051 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:23.861059 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 11:02:23.861123 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:02:23.861642 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:02:23.861643 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:02:23.861646 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:02:23.861694 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:23.861695 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:23.863022 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:23.863023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:23.864368 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:23.864371 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:23.865303 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:23.865304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:23.865375 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:23.865401 (+ 26us) client_negotiation.cc:769] Sending connection context
0929 11:02:23.865470 (+ 69us) client_negotiation.cc:240] Negotiation successful
0929 11:02:23.865526 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":40,"thread_start_us":72,"threads_started":1}
I20260929 11:02:23.865708 10729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:23.860883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:35161 (local address 127.0.0.1:58324)
0929 11:02:23.860938 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:23.860952 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 11:02:23.860994 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:02:23.861808 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:02:23.861814 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:02:23.861816 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:02:23.861854 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:23.861855 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:23.863242 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:23.863244 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:23.864386 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:23.864390 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:23.865465 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:23.865466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:23.865536 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:23.865540 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 11:02:23.865590 (+ 50us) client_negotiation.cc:240] Negotiation successful
0929 11:02:23.865640 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":2}
I20260929 11:02:23.865731 10998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:23.861242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58324 (local address 127.26.120.194:35161)
0929 11:02:23.861452 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0929 11:02:23.861456 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 11:02:23.861466 (+ 10us) server_negotiation.cc:407] Connection header received
0929 11:02:23.861507 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:02:23.861508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:02:23.861524 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:02:23.861612 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:02:23.861933 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:23.863162 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:23.864473 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:23.865216 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:23.865289 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:23.865344 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 11:02:23.865607 (+ 263us) server_negotiation.cc:299] Negotiation successful
0929 11:02:23.865650 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":98,"threads_started":1}
I20260929 11:02:23.865969 10999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:23.861210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55174 (local address 127.26.120.195:46867)
0929 11:02:23.861450 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0929 11:02:23.861455 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 11:02:23.861466 (+ 11us) server_negotiation.cc:407] Connection header received
0929 11:02:23.861506 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:02:23.861508 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:02:23.861524 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:02:23.861613 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:02:23.861782 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:23.862905 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:23.864498 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:23.865216 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:23.865287 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:23.865344 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 11:02:23.865828 (+ 484us) server_negotiation.cc:299] Negotiation successful
0929 11:02:23.865889 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":144,"threads_started":1}
I20260929 11:02:23.866473 10794 tablet_service.cc:1467] Processing CreateTablet for tablet 2499d7a661184028953313273842ba8d (DEFAULT_TABLE table=test-table [id=2585fce8d16a4aba96013b8b7822ebf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:02:23.866573 10926 tablet_service.cc:1467] Processing CreateTablet for tablet 2499d7a661184028953313273842ba8d (DEFAULT_TABLE table=test-table [id=2585fce8d16a4aba96013b8b7822ebf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:02:23.866829 10794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2499d7a661184028953313273842ba8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:02:23.866829 10926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2499d7a661184028953313273842ba8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:02:23.870366 11001 tablet_bootstrap.cc:492] T 2499d7a661184028953313273842ba8d P 46f46a759dfc42838abffdd16b09d2d7: Bootstrap starting.
I20260929 11:02:23.871124 11001 tablet_bootstrap.cc:654] T 2499d7a661184028953313273842ba8d P 46f46a759dfc42838abffdd16b09d2d7: Neither blocks nor log segments found. Creating new log.
I20260929 11:02:23.871585 11001 log.cc:824] T 2499d7a661184028953313273842ba8d P 46f46a759dfc42838abffdd16b09d2d7: Log is configured to *not* fsync() on all Append() calls
I20260929 11:02:23.871878 11002 tablet_bootstrap.cc:492] T 2499d7a661184028953313273842ba8d P 0d9c8dbb6e1e49a59e61c94c6c79e074: Bootstrap starting.
I20260929 11:02:23.872274 11001 tablet_bootstrap.cc:492] T 2499d7a661184028953313273842ba8d P 46f46a759dfc42838abffdd16b09d2d7: No bootstrap required, opened a new log
I20260929 11:02:23.872332 11001 ts_tablet_manager.cc:1397] T 2499d7a661184028953313273842ba8d P 46f46a759dfc42838abffdd16b09d2d7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 11:02:23.872489 11002 tablet_bootstrap.cc:654] T 2499d7a661184028953313273842ba8d P 0d9c8dbb6e1e49a59e61c94c6c79e074: Neither blocks nor log segments found. Creating new log.
I20260929 11:02:23.873010 11002 log.cc:824] T 2499d7a661184028953313273842ba8d P 0d9c8dbb6e1e49a59e61c94c6c79e074: Log is configured to *not* fsync() on all Append() calls
I20260929 11:02:23.873817 11002 tablet_bootstrap.cc:492] T 2499d7a661184028953313273842ba8d P 0d9c8dbb6e1e49a59e61c94c6c79e074: No bootstrap required, opened a new log
I20260929 11:02:23.873907 11002 ts_tablet_manager.cc:1397] T 2499d7a661184028953313273842ba8d P 0d9c8dbb6e1e49a59e61c94c6c79e074: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 11:02:23.874033 11001 raft_consensus.cc:348] T 2499d7a661184028953313273842ba8d P 46f46a759dfc42838abffdd16b09d2d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35161 } } peers { permanent_uuid: "46f46a759dfc42838abffdd16b09d2d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46867 } } peers { permanent_uuid: "20585d017b4b4bfb85a733ae593ebbeb" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44527 } }
I20260929 11:02:23.874298 11001 raft_consensus.cc:374] T 2499d7a661184028953313273842ba8d P 46f46a759dfc42838abffdd16b09d2d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:02:23.874341 11001 raft_consensus.cc:729] T 2499d7a661184028953313273842ba8d P 46f46a759dfc42838abffdd16b09d2d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46f46a759dfc42838abffdd16b09d2d7, State: Initialized, Role: FOLLOWER
I20260929 11:02:23.874456 11001 consensus_queue.cc:260] T 2499d7a661184028953313273842ba8d P 46f46a759dfc42838abffdd16b09d2d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35161 } } peers { permanent_uuid: "46f46a759dfc42838abffdd16b09d2d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46867 } } peers { permanent_uuid: "20585d017b4b4bfb85a733ae593ebbeb" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44527 } }
I20260929 11:02:23.874719 11001 ts_tablet_manager.cc:1428] T 2499d7a661184028953313273842ba8d P 46f46a759dfc42838abffdd16b09d2d7: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260929 11:02:23.874778 10992 heartbeater.cc:503] Master 127.26.120.254:44319 was elected leader, sending a full tablet report...
I20260929 11:02:23.876261 11002 raft_consensus.cc:348] T 2499d7a661184028953313273842ba8d P 0d9c8dbb6e1e49a59e61c94c6c79e074 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35161 } } peers { permanent_uuid: "46f46a759dfc42838abffdd16b09d2d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46867 } } peers { permanent_uuid: "20585d017b4b4bfb85a733ae593ebbeb" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44527 } }
I20260929 11:02:23.876407 11002 raft_consensus.cc:374] T 2499d7a661184028953313273842ba8d P 0d9c8dbb6e1e49a59e61c94c6c79e074 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:02:23.876431 11002 raft_consensus.cc:729] T 2499d7a661184028953313273842ba8d P 0d9c8dbb6e1e49a59e61c94c6c79e074 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9c8dbb6e1e49a59e61c94c6c79e074, State: Initialized, Role: FOLLOWER
I20260929 11:02:23.876552 11002 consensus_queue.cc:260] T 2499d7a661184028953313273842ba8d P 0d9c8dbb6e1e49a59e61c94c6c79e074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35161 } } peers { permanent_uuid: "46f46a759dfc42838abffdd16b09d2d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46867 } } peers { permanent_uuid: "20585d017b4b4bfb85a733ae593ebbeb" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44527 } }
I20260929 11:02:23.876812 11002 ts_tablet_manager.cc:1428] T 2499d7a661184028953313273842ba8d P 0d9c8dbb6e1e49a59e61c94c6c79e074: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 11:02:23.876842 10860 heartbeater.cc:503] Master 127.26.120.254:44319 was elected leader, sending a full tablet report...
W20260929 11:02:23.960114 10861 tablet.cc:2369] T 2499d7a661184028953313273842ba8d P 0d9c8dbb6e1e49a59e61c94c6c79e074: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 11:02:24.021067 10728 raft_consensus.cc:482] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 11:02:24.021211 10728 raft_consensus.cc:504] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35161 } } peers { permanent_uuid: "46f46a759dfc42838abffdd16b09d2d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46867 } } peers { permanent_uuid: "20585d017b4b4bfb85a733ae593ebbeb" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44527 } }
I20260929 11:02:24.021646 10728 leader_election.cc:288] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d9c8dbb6e1e49a59e61c94c6c79e074 (127.26.120.194:35161), 46f46a759dfc42838abffdd16b09d2d7 (127.26.120.195:46867)
I20260929 11:02:24.024712 10998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:24.021939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:36543 (local address 127.26.120.194:35161)
0929 11:02:24.021989 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 11:02:24.021992 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:02:24.022004 (+ 12us) server_negotiation.cc:407] Connection header received
0929 11:02:24.022093 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:02:24.022094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:02:24.022105 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:02:24.022164 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:02:24.022406 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:24.022947 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:24.023964 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:24.024344 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:24.024432 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:24.024443 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0929 11:02:24.024596 (+ 153us) server_negotiation.cc:299] Negotiation successful
0929 11:02:24.024637 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260929 11:02:24.024712 10724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:24.021854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:35161 (local address 127.26.120.193:36543)
0929 11:02:24.021908 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:24.021921 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 11:02:24.021981 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:02:24.022230 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:02:24.022232 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:02:24.022234 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:02:24.022305 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:24.022307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:24.023036 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:24.023037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:24.023862 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:24.023865 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:24.024432 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:24.024433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:24.024504 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:24.024521 (+ 17us) client_negotiation.cc:769] Sending connection context
0929 11:02:24.024566 (+ 45us) client_negotiation.cc:240] Negotiation successful
0929 11:02:24.024631 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260929 11:02:24.024734 10999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:24.022020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:57673 (local address 127.26.120.195:46867)
0929 11:02:24.022082 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0929 11:02:24.022084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:02:24.022206 (+ 122us) server_negotiation.cc:407] Connection header received
0929 11:02:24.022238 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:02:24.022239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:02:24.022250 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:02:24.022333 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:02:24.022541 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:24.023017 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:24.023963 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:24.024343 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:24.024433 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:24.024442 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 11:02:24.024631 (+ 189us) server_negotiation.cc:299] Negotiation successful
0929 11:02:24.024671 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 11:02:24.024806 11007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:24.021910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:46867 (local address 127.26.120.193:57673)
0929 11:02:24.022090 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:24.022100 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 11:02:24.022161 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:02:24.022387 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:02:24.022389 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:02:24.022391 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:02:24.022464 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:24.022466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:24.023105 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:24.023106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:24.023866 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:24.023868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:24.024452 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:24.024453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:24.024530 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:24.024534 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 11:02:24.024609 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 11:02:24.024645 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":1,"thread_start_us":92,"threads_started":1}
I20260929 11:02:24.025089 10946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2499d7a661184028953313273842ba8d" candidate_uuid: "20585d017b4b4bfb85a733ae593ebbeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46f46a759dfc42838abffdd16b09d2d7" is_pre_election: true
I20260929 11:02:24.025089 10814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2499d7a661184028953313273842ba8d" candidate_uuid: "20585d017b4b4bfb85a733ae593ebbeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" is_pre_election: true
I20260929 11:02:24.025241 10946 raft_consensus.cc:2454] T 2499d7a661184028953313273842ba8d P 46f46a759dfc42838abffdd16b09d2d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20585d017b4b4bfb85a733ae593ebbeb in term 0.
I20260929 11:02:24.025241 10814 raft_consensus.cc:2454] T 2499d7a661184028953313273842ba8d P 0d9c8dbb6e1e49a59e61c94c6c79e074 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20585d017b4b4bfb85a733ae593ebbeb in term 0.
I20260929 11:02:24.025463 10608 leader_election.cc:302] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20585d017b4b4bfb85a733ae593ebbeb, 46f46a759dfc42838abffdd16b09d2d7; no voters:
I20260929 11:02:24.025593 10728 raft_consensus.cc:2781] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 11:02:24.025652 10728 raft_consensus.cc:482] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 11:02:24.025684 10728 raft_consensus.cc:3037] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:02:24.026417 10728 raft_consensus.cc:504] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35161 } } peers { permanent_uuid: "46f46a759dfc42838abffdd16b09d2d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46867 } } peers { permanent_uuid: "20585d017b4b4bfb85a733ae593ebbeb" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44527 } }
I20260929 11:02:24.026559 10728 leader_election.cc:288] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [CANDIDATE]: Term 1 election: Requested vote from peers 0d9c8dbb6e1e49a59e61c94c6c79e074 (127.26.120.194:35161), 46f46a759dfc42838abffdd16b09d2d7 (127.26.120.195:46867)
I20260929 11:02:24.026711 10814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2499d7a661184028953313273842ba8d" candidate_uuid: "20585d017b4b4bfb85a733ae593ebbeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074"
I20260929 11:02:24.026728 10946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2499d7a661184028953313273842ba8d" candidate_uuid: "20585d017b4b4bfb85a733ae593ebbeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46f46a759dfc42838abffdd16b09d2d7"
I20260929 11:02:24.026803 10814 raft_consensus.cc:3037] T 2499d7a661184028953313273842ba8d P 0d9c8dbb6e1e49a59e61c94c6c79e074 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:02:24.026803 10946 raft_consensus.cc:3037] T 2499d7a661184028953313273842ba8d P 46f46a759dfc42838abffdd16b09d2d7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:02:24.027693 10946 raft_consensus.cc:2454] T 2499d7a661184028953313273842ba8d P 46f46a759dfc42838abffdd16b09d2d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20585d017b4b4bfb85a733ae593ebbeb in term 1.
I20260929 11:02:24.027693 10814 raft_consensus.cc:2454] T 2499d7a661184028953313273842ba8d P 0d9c8dbb6e1e49a59e61c94c6c79e074 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20585d017b4b4bfb85a733ae593ebbeb in term 1.
I20260929 11:02:24.027877 10611 leader_election.cc:302] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9c8dbb6e1e49a59e61c94c6c79e074, 20585d017b4b4bfb85a733ae593ebbeb; no voters:
I20260929 11:02:24.027985 10728 raft_consensus.cc:2781] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:02:24.028159 10728 raft_consensus.cc:686] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [term 1 LEADER]: Becoming Leader. State: Replica: 20585d017b4b4bfb85a733ae593ebbeb, State: Running, Role: LEADER
I20260929 11:02:24.028224 10728 consensus_queue.cc:237] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35161 } } peers { permanent_uuid: "46f46a759dfc42838abffdd16b09d2d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46867 } } peers { permanent_uuid: "20585d017b4b4bfb85a733ae593ebbeb" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44527 } }
I20260929 11:02:24.029045 10534 catalog_manager.cc:5495] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb reported cstate change: term changed from 0 to 1, leader changed from <none> to 20585d017b4b4bfb85a733ae593ebbeb (127.26.120.193). New cstate: current_term: 1 leader_uuid: "20585d017b4b4bfb85a733ae593ebbeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46f46a759dfc42838abffdd16b09d2d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20585d017b4b4bfb85a733ae593ebbeb" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44527 } health_report { overall_health: HEALTHY } } }
I20260929 11:02:24.047799 10730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:24.045391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44608 (local address 127.26.120.193:44527)
0929 11:02:24.045437 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 11:02:24.045439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:02:24.045448 (+ 9us) server_negotiation.cc:407] Connection header received
0929 11:02:24.045509 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:02:24.045510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:02:24.045528 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:02:24.045600 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 11:02:24.045741 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:24.046381 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:24.047152 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:24.047409 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:24.047476 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:24.047588 (+ 112us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 11:02:24.047633 (+ 45us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0929 11:02:24.047741 (+ 108us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44608: token expired
Metrics: {"server-negotiator.queue_time_us":19}
W20260929 11:02:24.047856 10730 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44608: token expired
I20260929 11:02:24.053622 10725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:24.048186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41646 (local address 127.26.120.254:44319)
0929 11:02:24.048250 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0929 11:02:24.048251 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 11:02:24.048258 (+ 7us) server_negotiation.cc:407] Connection header received
0929 11:02:24.048277 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:02:24.048278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:02:24.048285 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:02:24.048331 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:02:24.048808 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:24.049266 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:24.049973 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:24.050231 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:24.050296 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:24.051390 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:02:24.051391 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:02:24.051392 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:02:24.051398 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:02:24.052750 (+ 1352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:02:24.053045 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:02:24.053046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:02:24.053046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:02:24.053095 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:02:24.053254 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:02:24.053255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:02:24.053255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:02:24.053349 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:02:24.053421 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 11:02:24.053502 (+ 81us) server_negotiation.cc:299] Negotiation successful
0929 11:02:24.053563 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 11:02:24.056576 10730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:24.054527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44620 (local address 127.26.120.193:44527)
0929 11:02:24.054564 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 11:02:24.054566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:02:24.054573 (+ 7us) server_negotiation.cc:407] Connection header received
0929 11:02:24.054593 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:02:24.054593 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:02:24.054601 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:02:24.054655 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 11:02:24.054813 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:24.055236 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:24.055997 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:24.056175 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:24.056242 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:24.056340 (+ 98us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 11:02:24.056432 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 11:02:24.056468 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0929 11:02:24.056517 (+ 49us) server_negotiation.cc:299] Negotiation successful
0929 11:02:24.056530 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 11:02:24.058121 10946 raft_consensus.cc:1261] T 2499d7a661184028953313273842ba8d P 46f46a759dfc42838abffdd16b09d2d7 [term 1 FOLLOWER]: Refusing update from remote peer 20585d017b4b4bfb85a733ae593ebbeb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 11:02:24.058192 10814 raft_consensus.cc:1261] T 2499d7a661184028953313273842ba8d P 0d9c8dbb6e1e49a59e61c94c6c79e074 [term 1 FOLLOWER]: Refusing update from remote peer 20585d017b4b4bfb85a733ae593ebbeb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 11:02:24.058504 11008 consensus_queue.cc:1035] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "46f46a759dfc42838abffdd16b09d2d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 11:02:24.058634 10728 consensus_queue.cc:1035] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 11:02:24.060961 11011 mvcc.cc:205] Tried to move back new op lower bound from 7334624231659487232 to 7334624231540256768. Current Snapshot: MvccSnapshot[applied={T|T < 7334624231659487232}]
I20260929 11:02:24.061486 11013 mvcc.cc:205] Tried to move back new op lower bound from 7334624231659487232 to 7334624231540256768. Current Snapshot: MvccSnapshot[applied={T|T < 7334624231659487232 or (T in {7334624231659487232})}]
I20260929 11:02:24.062351 11012 mvcc.cc:205] Tried to move back new op lower bound from 7334624231659487232 to 7334624231540256768. Current Snapshot: MvccSnapshot[applied={T|T < 7334624231659487232 or (T in {7334624231659487232})}]
I20260929 11:02:24.063483 10534 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41646:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 11:02:24.063557 10534 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41646:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 11:02:24.064271 10534 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4: Sending DeleteTablet for 3 replicas of tablet 2499d7a661184028953313273842ba8d
I20260929 11:02:24.064586 10794 tablet_service.cc:1514] Processing DeleteTablet for tablet 2499d7a661184028953313273842ba8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:02:24 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:58324
I20260929 11:02:24.064618 10926 tablet_service.cc:1514] Processing DeleteTablet for tablet 2499d7a661184028953313273842ba8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:02:24 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:55174
I20260929 11:02:24.064658 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 10572
W20260929 11:02:24.069998 10518 connection.cc:535] client connection to 127.26.120.193:44527 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 11:02:24.070107 11021 tablet_replica.cc:330] T 2499d7a661184028953313273842ba8d P 46f46a759dfc42838abffdd16b09d2d7: stopping tablet replica
W20260929 11:02:24.070132 10518 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 11:02:24.070222 11021 raft_consensus.cc:2229] T 2499d7a661184028953313273842ba8d P 46f46a759dfc42838abffdd16b09d2d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 11:02:24.070235 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 10727
I20260929 11:02:24.070349 11021 raft_consensus.cc:2258] T 2499d7a661184028953313273842ba8d P 46f46a759dfc42838abffdd16b09d2d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 11:02:24.071703 11021 ts_tablet_manager.cc:1905] T 2499d7a661184028953313273842ba8d P 46f46a759dfc42838abffdd16b09d2d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 11:02:24.073875 11021 ts_tablet_manager.cc:1918] T 2499d7a661184028953313273842ba8d P 46f46a759dfc42838abffdd16b09d2d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 11:02:24.073954 11021 log.cc:1196] T 2499d7a661184028953313273842ba8d P 46f46a759dfc42838abffdd16b09d2d7: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/wal/wals/2499d7a661184028953313273842ba8d
I20260929 11:02:24.074272 11021 ts_tablet_manager.cc:1939] T 2499d7a661184028953313273842ba8d P 46f46a759dfc42838abffdd16b09d2d7: Deleting consensus metadata
I20260929 11:02:24.074486 10729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:24.074352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:44527 (local address 127.0.0.1:44630)
0929 11:02:24.074410 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:24.074457 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.193:44527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260929 11:02:24.074610 10518 catalog_manager.cc:4584] TS 20585d017b4b4bfb85a733ae593ebbeb (127.26.120.193:44527): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2499d7a661184028953313273842ba8d: Network error: Client connection negotiation failed: client connection to 127.26.120.193:44527: connect: Connection refused (error 111)
I20260929 11:02:24.075109 10518 catalog_manager.cc:4841] TS 46f46a759dfc42838abffdd16b09d2d7 (127.26.120.195:46867): tablet 2499d7a661184028953313273842ba8d (table test-table [id=2585fce8d16a4aba96013b8b7822ebf4]) successfully deleted
I20260929 11:02:24.075747 10729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:24.075657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:35161 (local address 127.0.0.1:58336)
0929 11:02:24.075703 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:24.075727 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.194:35161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260929 11:02:24.075794 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 10863
I20260929 11:02:24.081151 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 10501
I20260929 11:02:44.086560 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:44319
--webserver_interface=127.26.120.254
--webserver_port=36475
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:34649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:44319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:02:44.171270 11023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:02:44.171478 11023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:02:44.171499 11023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:02:44.172672 11023 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 11:02:44.172715 11023 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 11:02:44.172729 11023 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 11:02:44.172751 11023 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:02:44.172765 11023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:02:44.172777 11023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:02:44.172789 11023 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:02:44.172801 11023 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:02:44.174252 11023 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:34649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:44319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:44319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=36475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:02:44.174454 11023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:02:44.174628 11023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:02:44.180105 11031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:02:44.180114 11028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:02:44.180127 11029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:02:44.180401 11023 server_base.cc:1034] running on GCE node
I20260929 11:02:44.180673 11023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:02:44.180998 11023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:02:44.182127 11023 hybrid_clock.cc:648] HybridClock initialized: now 1790679764182107 us; error 30 us; skew 500 ppm
Sep 29 11:02:44 dist-test-slave-2lzq krb5kdc[9953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679764, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:02:44.194901 11023 init.cc:377] Logged in from keytab as kudu/127.26.120.254@KRBTEST.COM (short username kudu)
I20260929 11:02:44.195501 11023 webserver.cc:460] Webserver started at http://127.26.120.254:36475/ using document root <none> and password file <none>
I20260929 11:02:44.195765 11023 fs_manager.cc:359] Metadata directory not provided
I20260929 11:02:44.195837 11023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:02:44.197353 11023 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:02:44.198180 11038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:02:44.198410 11023 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:02:44.198521 11023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "037fbc5362fc4c5ab9f7ece66813ebe4"
format_stamp: "Formatted at 2026-09-29 11:02:02 on dist-test-slave-2lzq"
server_key: "307b583f309db733d33df0a955b7cbc0"
server_key_iv: "e6a04accb7f7dddea1607b03bbd44190"
server_key_version: "encryptionkey@0"
I20260929 11:02:44.198939 11023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:02:44.213447 11023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:02:44.215889 11023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:02:44.216070 11023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:02:44.220463 11023 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:44319
I20260929 11:02:44.220520 11090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:44319 every 8 connection(s)
I20260929 11:02:44.220932 11023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:02:44.222491 11091 sys_catalog.cc:263] Verifying existing consensus state
I20260929 11:02:44.222514 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 11023
I20260929 11:02:44.222826 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.193:44527
--local_ip_for_outbound_sockets=127.26.120.193
--tserver_master_addrs=127.26.120.254:44319
--webserver_port=44249
--webserver_interface=127.26.120.193
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:34649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 11:02:44.223307 11091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4: Bootstrap starting.
I20260929 11:02:44.231824 11091 log.cc:824] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4: Log is configured to *not* fsync() on all Append() calls
I20260929 11:02:44.234169 11091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 11:02:44.234580 11091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4: Bootstrap complete.
I20260929 11:02:44.236938 11091 raft_consensus.cc:348] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "037fbc5362fc4c5ab9f7ece66813ebe4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44319 } }
I20260929 11:02:44.237185 11091 raft_consensus.cc:729] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 037fbc5362fc4c5ab9f7ece66813ebe4, State: Initialized, Role: FOLLOWER
I20260929 11:02:44.237401 11091 consensus_queue.cc:260] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "037fbc5362fc4c5ab9f7ece66813ebe4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44319 } }
I20260929 11:02:44.237496 11091 raft_consensus.cc:388] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 11:02:44.237571 11091 raft_consensus.cc:482] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 11:02:44.237625 11091 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [term 2 FOLLOWER]: Advancing to term 3
I20260929 11:02:44.238708 11091 raft_consensus.cc:504] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "037fbc5362fc4c5ab9f7ece66813ebe4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44319 } }
I20260929 11:02:44.238833 11091 leader_election.cc:302] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 037fbc5362fc4c5ab9f7ece66813ebe4; no voters:
I20260929 11:02:44.239048 11091 leader_election.cc:288] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260929 11:02:44.239082 11095 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [term 3 FOLLOWER]: Leader election won for term 3
I20260929 11:02:44.239199 11095 raft_consensus.cc:686] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [term 3 LEADER]: Becoming Leader. State: Replica: 037fbc5362fc4c5ab9f7ece66813ebe4, State: Running, Role: LEADER
I20260929 11:02:44.239310 11091 sys_catalog.cc:564] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:02:44.239283 11095 consensus_queue.cc:237] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "037fbc5362fc4c5ab9f7ece66813ebe4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44319 } }
I20260929 11:02:44.239596 11097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 037fbc5362fc4c5ab9f7ece66813ebe4. Latest consensus state: current_term: 3 leader_uuid: "037fbc5362fc4c5ab9f7ece66813ebe4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "037fbc5362fc4c5ab9f7ece66813ebe4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44319 } } }
I20260929 11:02:44.239594 11096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "037fbc5362fc4c5ab9f7ece66813ebe4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "037fbc5362fc4c5ab9f7ece66813ebe4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44319 } } }
I20260929 11:02:44.239775 11096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [sys.catalog]: This master's current role is: LEADER
I20260929 11:02:44.239975 11097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4 [sys.catalog]: This master's current role is: LEADER
I20260929 11:02:44.240118 11101 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:02:44.240921 11101 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:02:44.241076 11101 catalog_manager.cc:1259] Loaded cluster ID: 61e7e82bb0c448e3b54e7fcdfde61c94
I20260929 11:02:44.241324 11101 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 11:02:44.242599 11101 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:02:44.242865 11101 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4: Loaded TSK: 0
I20260929 11:02:44.242969 11101 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 037fbc5362fc4c5ab9f7ece66813ebe4: Loaded TSK: 1
F20260929 11:02:44.243257 11101 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f817384e9ad google::LogMessage::Fail() at ??:0
@ 0x7f817385188a google::LogMessage::SendToLog() at ??:0
@ 0x7f817384e4de google::LogMessage::Flush() at ??:0
@ 0x7f817384fa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f8175fbb104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f8175fb8d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f817b7a81cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f817b7a848e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f817b7a8594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f817b76ec48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f817b778ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f817571c0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f8175711954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f81765756db start_thread at ??:0
@ 0x7f817234c61f clone at ??:0
W20260929 11:02:44.319298 11093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:02:44.319531 11093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:02:44.319568 11093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:02:44.320691 11093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:02:44.320760 11093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:02:44.320804 11093 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.193
I20260929 11:02:44.322353 11093 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:34649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.193:44527
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.120.193
--webserver_port=44249
--tserver_master_addrs=127.26.120.254:44319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.193
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:02:44.322613 11093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:02:44.322853 11093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:02:44.328960 11133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:02:44.329087 11135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:02:44.328967 11132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:02:44.329097 11093 server_base.cc:1034] running on GCE node
I20260929 11:02:44.329746 11093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:02:44.330003 11093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:02:44.331212 11093 hybrid_clock.cc:648] HybridClock initialized: now 1790679764331199 us; error 30 us; skew 500 ppm
Sep 29 11:02:44 dist-test-slave-2lzq krb5kdc[9953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679764, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:02:44.333748 11093 init.cc:377] Logged in from keytab as kudu/127.26.120.193@KRBTEST.COM (short username kudu)
I20260929 11:02:44.334331 11093 webserver.cc:460] Webserver started at http://127.26.120.193:44249/ using document root <none> and password file <none>
I20260929 11:02:44.334581 11093 fs_manager.cc:359] Metadata directory not provided
I20260929 11:02:44.334652 11093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:02:44.336023 11093 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:02:44.336727 11142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:02:44.336900 11093 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:02:44.336984 11093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/wal
uuid: "20585d017b4b4bfb85a733ae593ebbeb"
format_stamp: "Formatted at 2026-09-29 11:02:02 on dist-test-slave-2lzq"
server_key: "07c5bcb871d78c26fe0da942254088ac"
server_key_iv: "fb426bbcd89a9cdb68356ee7b1ba8e0c"
server_key_version: "encryptionkey@0"
I20260929 11:02:44.337360 11093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:02:44.348742 11093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:02:44.351334 11093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:02:44.351475 11093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:02:44.351715 11093 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:02:44.352201 11149 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260929 11:02:44.354151 11093 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 11:02:44.354223 11093 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260929 11:02:44.354267 11093 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 11:02:44.354974 11093 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 11:02:44.355031 11093 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260929 11:02:44.355098 11149 tablet_bootstrap.cc:492] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb: Bootstrap starting.
I20260929 11:02:44.362176 11093 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.193:44527
I20260929 11:02:44.362546 11093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
I20260929 11:02:44.362953 11256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.193:44527 every 8 connection(s)
I20260929 11:02:44.366115 11149 log.cc:824] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb: Log is configured to *not* fsync() on all Append() calls
I20260929 11:02:44.368026 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 11093
I20260929 11:02:44.368326 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.194:35161
--local_ip_for_outbound_sockets=127.26.120.194
--tserver_master_addrs=127.26.120.254:44319
--webserver_port=40263
--webserver_interface=127.26.120.194
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:34649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 11:02:44.368768 11149 tablet_bootstrap.cc:492] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 11:02:44.369071 11149 tablet_bootstrap.cc:492] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb: Bootstrap complete.
I20260929 11:02:44.370359 11149 ts_tablet_manager.cc:1397] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.010s
I20260929 11:02:44.371867 11149 raft_consensus.cc:348] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35161 } } peers { permanent_uuid: "46f46a759dfc42838abffdd16b09d2d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46867 } } peers { permanent_uuid: "20585d017b4b4bfb85a733ae593ebbeb" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44527 } }
I20260929 11:02:44.372215 11149 raft_consensus.cc:729] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20585d017b4b4bfb85a733ae593ebbeb, State: Initialized, Role: FOLLOWER
I20260929 11:02:44.372367 11149 consensus_queue.cc:260] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35161 } } peers { permanent_uuid: "46f46a759dfc42838abffdd16b09d2d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46867 } } peers { permanent_uuid: "20585d017b4b4bfb85a733ae593ebbeb" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44527 } }
I20260929 11:02:44.372624 11149 ts_tablet_manager.cc:1428] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
W20260929 11:02:44.441258 11259 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 11:02:44.363220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.193:56337)
0929 11:02:44.363461 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:44.363504 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 11:02:44.363955 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:02:44.441094 (+ 77139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":108,"threads_started":1}
W20260929 11:02:44.441543 11257 heartbeater.cc:644] Failed to heartbeat to 127.26.120.254:44319 (0 consecutive failures): Network error: Failed to ping master at 127.26.120.254:44319: Client connection negotiation failed: client connection to 127.26.120.254:44319: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 11:02:44.441905 11259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:44.441777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.193:54035)
0929 11:02:44.441826 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:44.441874 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260929 11:02:44.442291 11259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:44.442187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.193:38567)
0929 11:02:44.442237 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:44.442267 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260929 11:02:44.442416 11257 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 11:02:44.451676 11261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:02:44.451879 11261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:02:44.451902 11261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:02:44.453052 11261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:02:44.453094 11261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:02:44.453121 11261 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.194
I20260929 11:02:44.454684 11261 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:34649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.194:35161
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.120.194
--webserver_port=40263
--tserver_master_addrs=127.26.120.254:44319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.194
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:02:44.454923 11261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:02:44.455148 11261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:02:44.460322 11267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:02:44.460337 11270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:02:44.460342 11268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:02:44.460567 11261 server_base.cc:1034] running on GCE node
I20260929 11:02:44.460805 11261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:02:44.461087 11261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:02:44.462261 11261 hybrid_clock.cc:648] HybridClock initialized: now 1790679764462237 us; error 41 us; skew 500 ppm
Sep 29 11:02:44 dist-test-slave-2lzq krb5kdc[9953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679764, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:02:44.464880 11261 init.cc:377] Logged in from keytab as kudu/127.26.120.194@KRBTEST.COM (short username kudu)
I20260929 11:02:44.465349 11261 webserver.cc:460] Webserver started at http://127.26.120.194:40263/ using document root <none> and password file <none>
I20260929 11:02:44.465591 11261 fs_manager.cc:359] Metadata directory not provided
I20260929 11:02:44.465663 11261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:02:44.467155 11261 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:02:44.467887 11277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:02:44.468071 11261 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:02:44.468175 11261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/wal
uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074"
format_stamp: "Formatted at 2026-09-29 11:02:02 on dist-test-slave-2lzq"
server_key: "f9923899c0f03499f5bb1ff248a5c5e6"
server_key_iv: "28668a3585e22ba56b831a649a0e6206"
server_key_version: "encryptionkey@0"
I20260929 11:02:44.468538 11261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:02:44.483705 11261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:02:44.485699 11261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:02:44.485877 11261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:02:44.486132 11261 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:02:44.486572 11284 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260929 11:02:44.488035 11261 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 11:02:44.488101 11261 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260929 11:02:44.488145 11261 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 11:02:44.488844 11261 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 11:02:44.488899 11261 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260929 11:02:44.488991 11284 tablet_bootstrap.cc:492] T 2499d7a661184028953313273842ba8d P 0d9c8dbb6e1e49a59e61c94c6c79e074: Bootstrap starting.
I20260929 11:02:44.496235 11261 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.194:35161
I20260929 11:02:44.496330 11391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.194:35161 every 8 connection(s)
I20260929 11:02:44.496649 11261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
I20260929 11:02:44.497890 11394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:44.497357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.194:57533)
0929 11:02:44.497554 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:44.497666 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":75,"threads_started":1}
W20260929 11:02:44.498201 11392 heartbeater.cc:644] Failed to heartbeat to 127.26.120.254:44319 (0 consecutive failures): Network error: Failed to ping master at 127.26.120.254:44319: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
I20260929 11:02:44.498531 11394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:44.498420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.194:46199)
0929 11:02:44.498459 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:44.498497 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260929 11:02:44.498901 11394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:44.498791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.194:54121)
0929 11:02:44.498851 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:44.498879 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260929 11:02:44.499054 11392 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 11:02:44.499400 11284 log.cc:824] T 2499d7a661184028953313273842ba8d P 0d9c8dbb6e1e49a59e61c94c6c79e074: Log is configured to *not* fsync() on all Append() calls
I20260929 11:02:44.501436 11284 tablet_bootstrap.cc:492] T 2499d7a661184028953313273842ba8d P 0d9c8dbb6e1e49a59e61c94c6c79e074: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 11:02:44.501720 11284 tablet_bootstrap.cc:492] T 2499d7a661184028953313273842ba8d P 0d9c8dbb6e1e49a59e61c94c6c79e074: Bootstrap complete.
I20260929 11:02:44.502722 11284 ts_tablet_manager.cc:1397] T 2499d7a661184028953313273842ba8d P 0d9c8dbb6e1e49a59e61c94c6c79e074: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.006s
I20260929 11:02:44.503618 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 11261
I20260929 11:02:44.503914 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.195:46867
--local_ip_for_outbound_sockets=127.26.120.195
--tserver_master_addrs=127.26.120.254:44319
--webserver_port=41745
--webserver_interface=127.26.120.195
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:34649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 11:02:44.503973 11284 raft_consensus.cc:348] T 2499d7a661184028953313273842ba8d P 0d9c8dbb6e1e49a59e61c94c6c79e074 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35161 } } peers { permanent_uuid: "46f46a759dfc42838abffdd16b09d2d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46867 } } peers { permanent_uuid: "20585d017b4b4bfb85a733ae593ebbeb" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44527 } }
I20260929 11:02:44.504300 11284 raft_consensus.cc:729] T 2499d7a661184028953313273842ba8d P 0d9c8dbb6e1e49a59e61c94c6c79e074 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9c8dbb6e1e49a59e61c94c6c79e074, State: Initialized, Role: FOLLOWER
I20260929 11:02:44.504469 11284 consensus_queue.cc:260] T 2499d7a661184028953313273842ba8d P 0d9c8dbb6e1e49a59e61c94c6c79e074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35161 } } peers { permanent_uuid: "46f46a759dfc42838abffdd16b09d2d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46867 } } peers { permanent_uuid: "20585d017b4b4bfb85a733ae593ebbeb" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44527 } }
I20260929 11:02:44.504736 11284 ts_tablet_manager.cc:1428] T 2499d7a661184028953313273842ba8d P 0d9c8dbb6e1e49a59e61c94c6c79e074: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260929 11:02:44.504997 11394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:44.504894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.194:37939)
0929 11:02:44.504946 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:44.504972 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260929 11:02:44.505316 11394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:44.505223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.194:33933)
0929 11:02:44.505268 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:44.505291 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260929 11:02:44.587061 11397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:02:44.587256 11397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:02:44.587280 11397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:02:44.588436 11397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:02:44.588475 11397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:02:44.588502 11397 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.195
I20260929 11:02:44.589947 11397 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:34649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.195:46867
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.120.195
--webserver_port=41745
--tserver_master_addrs=127.26.120.254:44319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.195
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:02:44.590207 11397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:02:44.590443 11397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:02:44.595369 11402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:02:44.595418 11403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:02:44.595523 11397 server_base.cc:1034] running on GCE node
W20260929 11:02:44.595338 11405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:02:44.595815 11397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:02:44.596076 11397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:02:44.597280 11397 hybrid_clock.cc:648] HybridClock initialized: now 1790679764597262 us; error 27 us; skew 500 ppm
Sep 29 11:02:44 dist-test-slave-2lzq krb5kdc[9953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679764, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:02:44.599824 11397 init.cc:377] Logged in from keytab as kudu/127.26.120.195@KRBTEST.COM (short username kudu)
I20260929 11:02:44.600312 11397 webserver.cc:460] Webserver started at http://127.26.120.195:41745/ using document root <none> and password file <none>
I20260929 11:02:44.600564 11397 fs_manager.cc:359] Metadata directory not provided
I20260929 11:02:44.600636 11397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:02:44.601965 11397 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:02:44.602576 11412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:02:44.602749 11397 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 11:02:44.602855 11397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/wal
uuid: "46f46a759dfc42838abffdd16b09d2d7"
format_stamp: "Formatted at 2026-09-29 11:02:03 on dist-test-slave-2lzq"
server_key: "99ef178a55e0037763e2479557d74d62"
server_key_iv: "bc24c09cd7a2b69055dcf9ad767d3706"
server_key_version: "encryptionkey@0"
I20260929 11:02:44.603178 11397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:02:44.610582 11397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:02:44.612309 11397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:02:44.612465 11397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:02:44.612679 11397 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:02:44.612983 11397 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:02:44.613037 11397 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:02:44.613077 11397 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:02:44.613106 11397 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:02:44.619100 11397 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.195:46867
I20260929 11:02:44.619223 11525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.195:46867 every 8 connection(s)
I20260929 11:02:44.619539 11397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
I20260929 11:02:44.620612 11528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:44.620247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.195:51825)
0929 11:02:44.620405 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:44.620475 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":66,"threads_started":1}
W20260929 11:02:44.621040 11526 heartbeater.cc:644] Failed to heartbeat to 127.26.120.254:44319 (0 consecutive failures): Network error: Failed to ping master at 127.26.120.254:44319: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
I20260929 11:02:44.621311 11528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:44.621225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.195:60773)
0929 11:02:44.621270 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:44.621295 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260929 11:02:44.621632 11528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:44.621558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.195:43735)
0929 11:02:44.621597 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:44.621618 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260929 11:02:44.621802 11526 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 11:02:44.629248 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 11397
I20260929 11:02:45.442994 11530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:45.442756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.193:59983)
0929 11:02:45.442894 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:45.442931 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":63,"threads_started":1}
I20260929 11:02:45.506126 11531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:45.505833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.194:54101)
0929 11:02:45.505980 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:45.506014 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":85,"threads_started":1}
I20260929 11:02:45.531541 11532 raft_consensus.cc:482] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 11:02:45.531710 11532 raft_consensus.cc:504] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35161 } } peers { permanent_uuid: "46f46a759dfc42838abffdd16b09d2d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46867 } } peers { permanent_uuid: "20585d017b4b4bfb85a733ae593ebbeb" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44527 } }
I20260929 11:02:45.532102 11532 leader_election.cc:288] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d9c8dbb6e1e49a59e61c94c6c79e074 (127.26.120.194:35161), 46f46a759dfc42838abffdd16b09d2d7 (127.26.120.195:46867)
Sep 29 11:02:45 dist-test-slave-2lzq krb5kdc[9953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679764, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for kudu/127.26.120.195@KRBTEST.COM
Sep 29 11:02:45 dist-test-slave-2lzq krb5kdc[9953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679764, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for kudu/127.26.120.194@KRBTEST.COM
I20260929 11:02:45.542896 11534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:45.532438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:57707 (local address 127.26.120.194:35161)
0929 11:02:45.532560 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0929 11:02:45.532572 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 11:02:45.532598 (+ 26us) server_negotiation.cc:407] Connection header received
0929 11:02:45.532736 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:02:45.532738 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:02:45.532778 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:02:45.532917 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:02:45.533584 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:45.534533 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:45.535559 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:45.536154 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:45.536277 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:45.539087 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:02:45.539091 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:02:45.539092 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:02:45.539106 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:02:45.541167 (+ 2061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:02:45.541586 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:02:45.541589 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:02:45.541589 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:02:45.541699 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:02:45.541968 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:02:45.541969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:02:45.541970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:02:45.542281 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:02:45.542376 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0929 11:02:45.542655 (+ 279us) server_negotiation.cc:299] Negotiation successful
0929 11:02:45.542787 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":51,"threads_started":1}
I20260929 11:02:45.542898 11533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:45.532324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:46867 (local address 127.26.120.193:49339)
0929 11:02:45.532487 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:45.532498 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 11:02:45.532555 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:02:45.533260 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:02:45.533261 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:02:45.533477 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:02:45.533518 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:45.533519 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:45.534737 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:45.534738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:45.535503 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:45.535506 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:45.536271 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:45.536272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:45.536372 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:45.536977 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:02:45.536985 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:02:45.538533 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:02:45.541159 (+ 2626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:02:45.541167 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:02:45.541170 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:02:45.541493 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:02:45.541775 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:02:45.541776 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:02:45.541777 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:02:45.541897 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:02:45.542364 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:02:45.542365 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:02:45.542496 (+ 131us) client_negotiation.cc:769] Sending connection context
0929 11:02:45.542607 (+ 111us) client_negotiation.cc:240] Negotiation successful
0929 11:02:45.542802 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":96,"thread_start_us":87,"threads_started":1}
I20260929 11:02:45.542945 11535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:45.532451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:49339 (local address 127.26.120.195:46867)
0929 11:02:45.532614 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 11:02:45.532633 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0929 11:02:45.532651 (+ 18us) server_negotiation.cc:407] Connection header received
0929 11:02:45.533080 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:02:45.533083 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:02:45.533134 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:02:45.533279 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:02:45.533638 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:45.534622 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:45.535600 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:45.536181 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:45.536291 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:45.538654 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:02:45.538669 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:02:45.538669 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:02:45.538831 (+ 162us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:02:45.541063 (+ 2232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:02:45.541587 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:02:45.541590 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:02:45.541590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:02:45.541695 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:02:45.541971 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:02:45.541972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:02:45.541973 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:02:45.542281 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:02:45.542370 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 11:02:45.542657 (+ 287us) server_negotiation.cc:299] Negotiation successful
0929 11:02:45.542829 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":69,"threads_started":1}
I20260929 11:02:45.543012 11530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:45.532305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:35161 (local address 127.26.120.193:57707)
0929 11:02:45.532367 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:45.532379 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 11:02:45.532439 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:02:45.532951 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:02:45.532954 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:02:45.533281 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:02:45.533464 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:45.533469 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:45.534638 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:45.534639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:45.535463 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:45.535466 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:45.536252 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:45.536253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:45.536371 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:45.537060 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:02:45.537065 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:02:45.538954 (+ 1889us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:02:45.541262 (+ 2308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:02:45.541263 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:02:45.541264 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:02:45.541496 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:02:45.541790 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:02:45.541791 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:02:45.541791 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:02:45.541898 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:02:45.542357 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:02:45.542363 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:02:45.542496 (+ 133us) client_negotiation.cc:769] Sending connection context
0929 11:02:45.542606 (+ 110us) client_negotiation.cc:240] Negotiation successful
0929 11:02:45.542884 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":133}
W20260929 11:02:45.543751 11143 leader_election.cc:341] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46f46a759dfc42838abffdd16b09d2d7 (127.26.120.195:46867): Not found: Tablet not found: 2499d7a661184028953313273842ba8d
I20260929 11:02:45.543948 11346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2499d7a661184028953313273842ba8d" candidate_uuid: "20585d017b4b4bfb85a733ae593ebbeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" is_pre_election: true
I20260929 11:02:45.544112 11346 raft_consensus.cc:2454] T 2499d7a661184028953313273842ba8d P 0d9c8dbb6e1e49a59e61c94c6c79e074 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20585d017b4b4bfb85a733ae593ebbeb in term 1.
I20260929 11:02:45.544346 11146 leader_election.cc:302] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0d9c8dbb6e1e49a59e61c94c6c79e074, 20585d017b4b4bfb85a733ae593ebbeb; no voters: 46f46a759dfc42838abffdd16b09d2d7
I20260929 11:02:45.544464 11532 raft_consensus.cc:2781] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260929 11:02:45.544523 11532 raft_consensus.cc:482] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 11:02:45.544538 11532 raft_consensus.cc:3037] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [term 1 FOLLOWER]: Advancing to term 2
I20260929 11:02:45.545675 11532 raft_consensus.cc:504] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35161 } } peers { permanent_uuid: "46f46a759dfc42838abffdd16b09d2d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46867 } } peers { permanent_uuid: "20585d017b4b4bfb85a733ae593ebbeb" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44527 } }
I20260929 11:02:45.545823 11532 leader_election.cc:288] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [CANDIDATE]: Term 2 election: Requested vote from peers 0d9c8dbb6e1e49a59e61c94c6c79e074 (127.26.120.194:35161), 46f46a759dfc42838abffdd16b09d2d7 (127.26.120.195:46867)
W20260929 11:02:45.546088 11143 leader_election.cc:341] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 46f46a759dfc42838abffdd16b09d2d7 (127.26.120.195:46867): Not found: Tablet not found: 2499d7a661184028953313273842ba8d
I20260929 11:02:45.546036 11346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2499d7a661184028953313273842ba8d" candidate_uuid: "20585d017b4b4bfb85a733ae593ebbeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074"
I20260929 11:02:45.546164 11346 raft_consensus.cc:3037] T 2499d7a661184028953313273842ba8d P 0d9c8dbb6e1e49a59e61c94c6c79e074 [term 1 FOLLOWER]: Advancing to term 2
I20260929 11:02:45.547334 11346 raft_consensus.cc:2454] T 2499d7a661184028953313273842ba8d P 0d9c8dbb6e1e49a59e61c94c6c79e074 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20585d017b4b4bfb85a733ae593ebbeb in term 2.
I20260929 11:02:45.547554 11146 leader_election.cc:302] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0d9c8dbb6e1e49a59e61c94c6c79e074, 20585d017b4b4bfb85a733ae593ebbeb; no voters: 46f46a759dfc42838abffdd16b09d2d7
I20260929 11:02:45.547649 11532 raft_consensus.cc:2781] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [term 2 FOLLOWER]: Leader election won for term 2
I20260929 11:02:45.547782 11532 raft_consensus.cc:686] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [term 2 LEADER]: Becoming Leader. State: Replica: 20585d017b4b4bfb85a733ae593ebbeb, State: Running, Role: LEADER
I20260929 11:02:45.547881 11532 consensus_queue.cc:237] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35161 } } peers { permanent_uuid: "46f46a759dfc42838abffdd16b09d2d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46867 } } peers { permanent_uuid: "20585d017b4b4bfb85a733ae593ebbeb" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44527 } }
I20260929 11:02:45.548134 11533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:45.548042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.193:51059)
0929 11:02:45.548076 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:45.548102 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260929 11:02:45.622406 11538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:45.622169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.195:47745)
0929 11:02:45.622320 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:45.622366 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":79,"threads_started":1}
W20260929 11:02:46.079334 11143 consensus_peers.cc:487] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb -> Peer 46f46a759dfc42838abffdd16b09d2d7 (127.26.120.195:46867): Couldn't send request to peer 46f46a759dfc42838abffdd16b09d2d7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2499d7a661184028953313273842ba8d. This is attempt 1: this message will repeat every 5th retry.
I20260929 11:02:46.158563 11346 raft_consensus.cc:1261] T 2499d7a661184028953313273842ba8d P 0d9c8dbb6e1e49a59e61c94c6c79e074 [term 2 FOLLOWER]: Refusing update from remote peer 20585d017b4b4bfb85a733ae593ebbeb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260929 11:02:46.158954 11539 consensus_queue.cc:1035] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260929 11:02:46.159986 11542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:46.159770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.194:32839)
0929 11:02:46.159889 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:46.159939 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":48,"threads_started":1}
I20260929 11:02:46.160854 11546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:46.160670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.193:51089)
0929 11:02:46.160776 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:46.160807 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":46,"threads_started":1}
I20260929 11:02:46.160885 11542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:46.160798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.194:41527)
0929 11:02:46.160833 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:46.160860 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260929 11:02:46.622997 11547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:46.622782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.195:52887)
0929 11:02:46.622914 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:46.622955 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":59,"threads_started":1}
I20260929 11:02:46.650023 11548 ts_tablet_manager.cc:927] T 2499d7a661184028953313273842ba8d P 46f46a759dfc42838abffdd16b09d2d7: Initiating tablet copy from peer 20585d017b4b4bfb85a733ae593ebbeb (127.26.120.193:44527)
I20260929 11:02:46.650408 11548 tablet_copy_client.cc:316] T 2499d7a661184028953313273842ba8d P 46f46a759dfc42838abffdd16b09d2d7: tablet copy: Beginning tablet copy session from remote peer at address 127.26.120.193:44527
Sep 29 11:02:46 dist-test-slave-2lzq krb5kdc[9953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679764, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for kudu/127.26.120.193@KRBTEST.COM
I20260929 11:02:46.658516 11547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:46.650710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:44527 (local address 127.26.120.195:52537)
0929 11:02:46.650776 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:46.650816 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 11:02:46.650886 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:02:46.651148 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:02:46.651154 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:02:46.651481 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:02:46.651545 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:46.651548 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:46.652440 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:46.652441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:46.653052 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:02:46.653054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:02:46.653572 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:02:46.653573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:02:46.653652 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:46.654248 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:02:46.654253 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:02:46.655633 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:02:46.657307 (+ 1674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:02:46.657314 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:02:46.657316 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:02:46.657520 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:02:46.657756 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:02:46.657758 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:02:46.657758 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:02:46.657825 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:02:46.658163 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:02:46.658170 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:02:46.658226 (+ 56us) client_negotiation.cc:769] Sending connection context
0929 11:02:46.658314 (+ 88us) client_negotiation.cc:240] Negotiation successful
0929 11:02:46.658445 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260929 11:02:46.658519 11550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:46.650830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:52537 (local address 127.26.120.193:44527)
0929 11:02:46.651012 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 11:02:46.651017 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 11:02:46.651026 (+ 9us) server_negotiation.cc:407] Connection header received
0929 11:02:46.651049 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:02:46.651051 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:02:46.651063 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:02:46.651146 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:02:46.651639 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:46.652341 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:46.653143 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:02:46.653462 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:02:46.653544 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:02:46.655733 (+ 2189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:02:46.655741 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:02:46.655742 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:02:46.655749 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:02:46.657138 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:02:46.657618 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:02:46.657621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:02:46.657621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:02:46.657686 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:02:46.657900 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:02:46.657901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:02:46.657901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:02:46.658100 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:02:46.658174 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 11:02:46.658371 (+ 197us) server_negotiation.cc:299] Negotiation successful
0929 11:02:46.658450 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":100,"threads_started":1}
I20260929 11:02:46.659034 11231 tablet_copy_service.cc:140] P 20585d017b4b4bfb85a733ae593ebbeb: Received BeginTabletCopySession request for tablet 2499d7a661184028953313273842ba8d from peer 46f46a759dfc42838abffdd16b09d2d7 ({username='kudu', principal='kudu/127.26.120.195@KRBTEST.COM'} at 127.26.120.195:52537)
I20260929 11:02:46.659129 11231 tablet_copy_service.cc:161] P 20585d017b4b4bfb85a733ae593ebbeb: Beginning new tablet copy session on tablet 2499d7a661184028953313273842ba8d from peer 46f46a759dfc42838abffdd16b09d2d7 at {username='kudu', principal='kudu/127.26.120.195@KRBTEST.COM'} at 127.26.120.195:52537: session id = 46f46a759dfc42838abffdd16b09d2d7-2499d7a661184028953313273842ba8d
I20260929 11:02:46.659611 11231 tablet_copy_source_session.cc:205] T 2499d7a661184028953313273842ba8d P 20585d017b4b4bfb85a733ae593ebbeb: Tablet Copy: opened 0 blocks and 1 log segments
I20260929 11:02:46.660604 11548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2499d7a661184028953313273842ba8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:02:46.663383 11548 tablet_copy_client.cc:791] T 2499d7a661184028953313273842ba8d P 46f46a759dfc42838abffdd16b09d2d7: tablet copy: Starting download of 0 data blocks...
I20260929 11:02:46.663565 11548 tablet_copy_client.cc:655] T 2499d7a661184028953313273842ba8d P 46f46a759dfc42838abffdd16b09d2d7: tablet copy: Starting download of 1 WAL segments...
I20260929 11:02:46.664453 11548 tablet_copy_client.cc:531] T 2499d7a661184028953313273842ba8d P 46f46a759dfc42838abffdd16b09d2d7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260929 11:02:46.665102 11548 tablet_bootstrap.cc:492] T 2499d7a661184028953313273842ba8d P 46f46a759dfc42838abffdd16b09d2d7: Bootstrap starting.
I20260929 11:02:46.666710 11548 log.cc:824] T 2499d7a661184028953313273842ba8d P 46f46a759dfc42838abffdd16b09d2d7: Log is configured to *not* fsync() on all Append() calls
I20260929 11:02:46.668519 11548 tablet_bootstrap.cc:492] T 2499d7a661184028953313273842ba8d P 46f46a759dfc42838abffdd16b09d2d7: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 11:02:46.668699 11548 tablet_bootstrap.cc:492] T 2499d7a661184028953313273842ba8d P 46f46a759dfc42838abffdd16b09d2d7: Bootstrap complete.
I20260929 11:02:46.668789 11548 ts_tablet_manager.cc:1397] T 2499d7a661184028953313273842ba8d P 46f46a759dfc42838abffdd16b09d2d7: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260929 11:02:46.669685 11548 raft_consensus.cc:348] T 2499d7a661184028953313273842ba8d P 46f46a759dfc42838abffdd16b09d2d7 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35161 } } peers { permanent_uuid: "46f46a759dfc42838abffdd16b09d2d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46867 } } peers { permanent_uuid: "20585d017b4b4bfb85a733ae593ebbeb" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44527 } }
I20260929 11:02:46.669821 11548 raft_consensus.cc:729] T 2499d7a661184028953313273842ba8d P 46f46a759dfc42838abffdd16b09d2d7 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46f46a759dfc42838abffdd16b09d2d7, State: Initialized, Role: FOLLOWER
I20260929 11:02:46.669915 11548 consensus_queue.cc:260] T 2499d7a661184028953313273842ba8d P 46f46a759dfc42838abffdd16b09d2d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9c8dbb6e1e49a59e61c94c6c79e074" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 35161 } } peers { permanent_uuid: "46f46a759dfc42838abffdd16b09d2d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 46867 } } peers { permanent_uuid: "20585d017b4b4bfb85a733ae593ebbeb" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44527 } }
I20260929 11:02:46.670176 11548 ts_tablet_manager.cc:1428] T 2499d7a661184028953313273842ba8d P 46f46a759dfc42838abffdd16b09d2d7: Time spent starting tablet: real 0.001s user 0.002s sys 0.001s
I20260929 11:02:46.670440 11547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:46.670337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.195:34273)
0929 11:02:46.670388 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:46.670418 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260929 11:02:46.670533 11231 tablet_copy_service.cc:342] P 20585d017b4b4bfb85a733ae593ebbeb: Request end of tablet copy session 46f46a759dfc42838abffdd16b09d2d7-2499d7a661184028953313273842ba8d received from {username='kudu', principal='kudu/127.26.120.195@KRBTEST.COM'} at 127.26.120.195:52537
I20260929 11:02:46.670599 11231 tablet_copy_service.cc:433] P 20585d017b4b4bfb85a733ae593ebbeb: ending tablet copy session 46f46a759dfc42838abffdd16b09d2d7-2499d7a661184028953313273842ba8d on tablet 2499d7a661184028953313273842ba8d with peer 46f46a759dfc42838abffdd16b09d2d7
I20260929 11:02:46.670964 11547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:46.670867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.195:56119)
0929 11:02:46.670913 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:46.670941 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260929 11:02:47.133865 11555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:47.133673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.193:44095)
0929 11:02:47.133776 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:47.133803 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":49,"threads_started":1}
I20260929 11:02:47.133857 11480 raft_consensus.cc:1203] T 2499d7a661184028953313273842ba8d P 46f46a759dfc42838abffdd16b09d2d7 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260929 11:02:47.134327 11547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:47.134219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.195:32979)
0929 11:02:47.134269 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:47.134301 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260929 11:02:47.161402 11556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:47.161208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.194:52603)
0929 11:02:47.161321 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:47.161357 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":72,"threads_started":1}
I20260929 11:02:48.134819 11557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:48.134479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.193:51857)
0929 11:02:48.134697 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:48.134752 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":65,"threads_started":1}
I20260929 11:02:48.135102 11558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:48.134823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.195:39765)
0929 11:02:48.135018 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:48.135061 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":62,"threads_started":1}
I20260929 11:02:48.162307 11559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:48.162007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.194:54589)
0929 11:02:48.162194 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:48.162259 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":122,"threads_started":1}
I20260929 11:02:49.135596 11560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:49.135282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.193:35439)
0929 11:02:49.135500 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:49.135540 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":104,"threads_started":1}
I20260929 11:02:49.135774 11561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:49.135573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.195:51497)
0929 11:02:49.135683 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:49.135723 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":65,"threads_started":1}
I20260929 11:02:49.163019 11562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:49.162810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.194:37825)
0929 11:02:49.162936 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:49.162974 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":65,"threads_started":1}
I20260929 11:02:50.136251 11563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:50.136038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.193:34047)
0929 11:02:50.136172 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:50.136203 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":77,"threads_started":1}
I20260929 11:02:50.136458 11564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:50.136229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.195:45053)
0929 11:02:50.136349 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:50.136388 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":63,"threads_started":1}
I20260929 11:02:50.136664 11563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:50.136539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.193:42635)
0929 11:02:50.136610 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:50.136637 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260929 11:02:50.163779 11565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:50.163555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.194:40999)
0929 11:02:50.163677 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:50.163722 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":60,"threads_started":1}
I20260929 11:02:51.137213 11566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:51.136989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.195:50647)
0929 11:02:51.137121 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:51.137157 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":80,"threads_started":1}
I20260929 11:02:51.137383 11567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:51.137088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.193:53219)
0929 11:02:51.137288 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:51.137328 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":70,"threads_started":1}
I20260929 11:02:51.164433 11568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:51.164211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.194:56525)
0929 11:02:51.164348 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:51.164387 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":77,"threads_started":1}
I20260929 11:02:52.138091 11570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:52.137849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.193:58673)
0929 11:02:52.137976 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:52.138008 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":61,"threads_started":1}
I20260929 11:02:52.138128 11569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:52.137803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.195:57367)
0929 11:02:52.137980 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:52.138022 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":96,"threads_started":1}
I20260929 11:02:52.165134 11571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:52.164931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.194:54397)
0929 11:02:52.165048 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:52.165087 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":63,"threads_started":1}
I20260929 11:02:53.138895 11573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:53.138646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.195:56109)
0929 11:02:53.138809 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:53.138848 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":77,"threads_started":1}
I20260929 11:02:53.138918 11574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:53.138646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.193:34139)
0929 11:02:53.138830 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:53.138876 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":74,"threads_started":1}
I20260929 11:02:53.165841 11575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:53.165607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.194:38001)
0929 11:02:53.165759 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:53.165796 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":63,"threads_started":1}
I20260929 11:02:54.139557 11576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:54.139334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.193:53799)
0929 11:02:54.139469 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:54.139508 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
I20260929 11:02:54.139575 11577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:54.139321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.195:52039)
0929 11:02:54.139503 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:54.139533 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":81,"threads_started":1}
I20260929 11:02:54.166483 11578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:54.166285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.194:37821)
0929 11:02:54.166401 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:54.166438 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":72,"threads_started":1}
I20260929 11:02:55.140228 11579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:55.139998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.193:53645)
0929 11:02:55.140145 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:55.140182 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":61,"threads_started":1}
I20260929 11:02:55.140267 11580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:55.140022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.195:60073)
0929 11:02:55.140191 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:55.140226 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":73,"threads_started":1}
I20260929 11:02:55.167157 11581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:55.166953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.194:36205)
0929 11:02:55.167080 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:55.167111 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
I20260929 11:02:56.141000 11582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:56.140729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.193:37731)
0929 11:02:56.140888 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:56.140932 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":76,"threads_started":1}
I20260929 11:02:56.141075 11583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:56.140845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.195:58267)
0929 11:02:56.140965 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:56.141024 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":66,"threads_started":1}
I20260929 11:02:56.167913 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:56.167672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.194:48191)
0929 11:02:56.167827 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:56.167866 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":87,"threads_started":1}
I20260929 11:02:57.141667 11585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:57.141436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.193:51471)
0929 11:02:57.141585 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:57.141620 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":60,"threads_started":1}
I20260929 11:02:57.141702 11586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:57.141456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.195:57789)
0929 11:02:57.141596 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:57.141648 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":70,"threads_started":1}
I20260929 11:02:57.168550 11587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:57.168362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.194:37099)
0929 11:02:57.168475 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:57.168505 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":63,"threads_started":1}
I20260929 11:02:58.142354 11588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:58.142125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.193:60439)
0929 11:02:58.142261 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:58.142306 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":73,"threads_started":1}
I20260929 11:02:58.142354 11589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:58.142148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.195:39569)
0929 11:02:58.142271 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:58.142313 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":54,"threads_started":1}
I20260929 11:02:58.169178 11590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:58.168967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.194:57059)
0929 11:02:58.169096 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:58.169133 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":62,"threads_started":1}
I20260929 11:02:59.143098 11592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:59.142833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.193:44459)
0929 11:02:59.143012 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:59.143044 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":81,"threads_started":1}
I20260929 11:02:59.143105 11591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:59.142830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.195:59621)
0929 11:02:59.143027 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:59.143062 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":88,"threads_started":1}
I20260929 11:02:59.169866 11593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:02:59.169634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44319 (local address 127.26.120.194:43421)
0929 11:02:59.169776 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0929 11:02:59.169818 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.254:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":59,"threads_started":1}
I20260929 11:02:59.634600 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 11093
I20260929 11:02:59.639541 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 11261
I20260929 11:02:59.644510 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 11397
2026-09-29T11:02:59Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58320 ms)
[----------] 30 tests from SecurityITest (229664 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 11:02:59 dist-test-slave-2lzq krb5kdc[11596](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 11:02:59 dist-test-slave-2lzq krb5kdc[11596](info): set up 2 sockets
Sep 29 11:02:59 dist-test-slave-2lzq krb5kdc[11596](info): commencing operation
krb5kdc: starting...
W20260929 11:03:00.705969 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 11:03:00 dist-test-slave-2lzq krb5kdc[11596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T11:03:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T11:03:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.120.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
I20260929 11:03:00.837097 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:46241
--webserver_interface=127.26.120.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:42317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:46241
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:03:00.919956 11609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:00.920197 11609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:00.920233 11609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:00.921388 11609 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:03:00.921456 11609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:00.921479 11609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:00.921509 11609 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:03:00.921548 11609 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:03:00.923019 11609 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:42317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:46241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:46241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:00.923285 11609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:00.923512 11609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:00.928716 11617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:00.928799 11609 server_base.cc:1034] running on GCE node
W20260929 11:03:00.928786 11615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:00.928716 11614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:00.929144 11609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:00.929385 11609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:00.930613 11609 hybrid_clock.cc:648] HybridClock initialized: now 1790679780930589 us; error 39 us; skew 500 ppm
Sep 29 11:03:00 dist-test-slave-2lzq krb5kdc[11596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679780, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:03:00.933212 11609 init.cc:377] Logged in from keytab as kudu/127.26.120.254@KRBTEST.COM (short username kudu)
I20260929 11:03:00.933673 11609 webserver.cc:460] Webserver started at http://127.26.120.254:42749/ using document root <none> and password file <none>
I20260929 11:03:00.933887 11609 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:00.933928 11609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:00.934016 11609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:00.935074 11609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "17abfdfaa64c4f08855087a3405369b3"
format_stamp: "Formatted at 2026-09-29 11:03:00 on dist-test-slave-2lzq"
server_key: "9b677784db8d4606d96f72744e6d8e7f"
server_key_iv: "37bdbd9b960ea51b30afd3830b211fe8"
server_key_version: "encryptionkey@0"
I20260929 11:03:00.935375 11609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "17abfdfaa64c4f08855087a3405369b3"
format_stamp: "Formatted at 2026-09-29 11:03:00 on dist-test-slave-2lzq"
server_key: "9b677784db8d4606d96f72744e6d8e7f"
server_key_iv: "37bdbd9b960ea51b30afd3830b211fe8"
server_key_version: "encryptionkey@0"
I20260929 11:03:00.936602 11609 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 11:03:00.937338 11624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:00.937513 11609 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:03:00.937603 11609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "17abfdfaa64c4f08855087a3405369b3"
format_stamp: "Formatted at 2026-09-29 11:03:00 on dist-test-slave-2lzq"
server_key: "9b677784db8d4606d96f72744e6d8e7f"
server_key_iv: "37bdbd9b960ea51b30afd3830b211fe8"
server_key_version: "encryptionkey@0"
I20260929 11:03:00.937659 11609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:00.966032 11609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:00.968417 11609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:00.968533 11609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:00.972319 11609 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:46241
I20260929 11:03:00.972388 11676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:46241 every 8 connection(s)
I20260929 11:03:00.972766 11609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:03:00.973649 11677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:00.975951 11677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17abfdfaa64c4f08855087a3405369b3: Bootstrap starting.
I20260929 11:03:00.976543 11677 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17abfdfaa64c4f08855087a3405369b3: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:00.976824 11677 log.cc:824] T 00000000000000000000000000000000 P 17abfdfaa64c4f08855087a3405369b3: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:00.977551 11677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17abfdfaa64c4f08855087a3405369b3: No bootstrap required, opened a new log
I20260929 11:03:00.979167 11677 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17abfdfaa64c4f08855087a3405369b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17abfdfaa64c4f08855087a3405369b3" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 46241 } }
I20260929 11:03:00.979331 11677 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17abfdfaa64c4f08855087a3405369b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:00.979383 11677 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17abfdfaa64c4f08855087a3405369b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17abfdfaa64c4f08855087a3405369b3, State: Initialized, Role: FOLLOWER
I20260929 11:03:00.979563 11677 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17abfdfaa64c4f08855087a3405369b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17abfdfaa64c4f08855087a3405369b3" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 46241 } }
I20260929 11:03:00.979660 11677 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17abfdfaa64c4f08855087a3405369b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 11:03:00.979704 11677 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17abfdfaa64c4f08855087a3405369b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 11:03:00.979758 11677 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17abfdfaa64c4f08855087a3405369b3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:00.980283 11677 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17abfdfaa64c4f08855087a3405369b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17abfdfaa64c4f08855087a3405369b3" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 46241 } }
I20260929 11:03:00.980399 11677 leader_election.cc:302] T 00000000000000000000000000000000 P 17abfdfaa64c4f08855087a3405369b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17abfdfaa64c4f08855087a3405369b3; no voters:
I20260929 11:03:00.980592 11677 leader_election.cc:288] T 00000000000000000000000000000000 P 17abfdfaa64c4f08855087a3405369b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 11:03:00.980651 11680 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17abfdfaa64c4f08855087a3405369b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:03:00.980844 11677 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17abfdfaa64c4f08855087a3405369b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:03:00.980880 11680 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17abfdfaa64c4f08855087a3405369b3 [term 1 LEADER]: Becoming Leader. State: Replica: 17abfdfaa64c4f08855087a3405369b3, State: Running, Role: LEADER
I20260929 11:03:00.980984 11680 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17abfdfaa64c4f08855087a3405369b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17abfdfaa64c4f08855087a3405369b3" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 46241 } }
I20260929 11:03:00.981379 11682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17abfdfaa64c4f08855087a3405369b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17abfdfaa64c4f08855087a3405369b3. Latest consensus state: current_term: 1 leader_uuid: "17abfdfaa64c4f08855087a3405369b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17abfdfaa64c4f08855087a3405369b3" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 46241 } } }
I20260929 11:03:00.981464 11682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17abfdfaa64c4f08855087a3405369b3 [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:00.981513 11681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17abfdfaa64c4f08855087a3405369b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "17abfdfaa64c4f08855087a3405369b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17abfdfaa64c4f08855087a3405369b3" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 46241 } } }
I20260929 11:03:00.981583 11681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17abfdfaa64c4f08855087a3405369b3 [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:00.982450 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 11609
I20260929 11:03:00.982558 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 11:03:00.982754 27107 external_mini_cluster.cc:1428] Setting key b14d5daef1a76c2cf345585e6447a455
W20260929 11:03:00.982758 11696 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 17abfdfaa64c4f08855087a3405369b3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 11:03:00.982801 11696 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 11:03:00.982856 11689 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:03:00.982998 11689 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:03:00.984690 11689 catalog_manager.cc:1347] Generated new cluster ID: ae3a9821f687413badcd9014c39965a3
I20260929 11:03:00.984781 11689 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 11:03:00 dist-test-slave-2lzq krb5kdc[11596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:03:00.995014 11689 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 11:03:00.995699 11689 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:03:00.997251 11699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:00.983206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33926 (local address 127.26.120.254:46241)
0929 11:03:00.986173 (+ 2967us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:00.986183 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:00.986220 (+ 37us) server_negotiation.cc:407] Connection header received
0929 11:03:00.986350 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:00.986357 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:00.986432 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:00.986590 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:00.987328 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:00.988789 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:00.989851 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:00.990520 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:00.990643 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:00.993408 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:00.993412 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:00.993413 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:00.993425 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:00.995613 (+ 2188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:00.996029 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:00.996032 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:00.996032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:00.996134 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:00.996352 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:00.996353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:00.996353 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:00.996616 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:00.996742 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:00.996863 (+ 121us) server_negotiation.cc:299] Negotiation successful
0929 11:03:00.996999 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2907,"thread_start_us":79,"threads_started":1}
I20260929 11:03:01.004431 11689 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17abfdfaa64c4f08855087a3405369b3: Generated new TSK 0
I20260929 11:03:01.004614 11689 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.120.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab.
Entry for principal kudu/127.26.120.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab.
I20260929 11:03:01.031213 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.193:0
--local_ip_for_outbound_sockets=127.26.120.193
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:46241
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:42317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:03:01.111019 11703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:01.111260 11703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:01.111299 11703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:01.112435 11703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:01.112500 11703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:01.112543 11703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.193
I20260929 11:03:01.114002 11703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:42317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:46241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.193
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:01.114423 11703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:01.114780 11703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:01.120113 11708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:01.120113 11709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:01.120106 11711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:01.120133 11703 server_base.cc:1034] running on GCE node
I20260929 11:03:01.120517 11703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:01.120769 11703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:01.121941 11703 hybrid_clock.cc:648] HybridClock initialized: now 1790679781121918 us; error 35 us; skew 500 ppm
Sep 29 11:03:01 dist-test-slave-2lzq krb5kdc[11596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679781, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:03:01.124610 11703 init.cc:377] Logged in from keytab as kudu/127.26.120.193@KRBTEST.COM (short username kudu)
I20260929 11:03:01.125135 11703 webserver.cc:460] Webserver started at http://127.26.120.193:36277/ using document root <none> and password file <none>
I20260929 11:03:01.125331 11703 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:01.125371 11703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:01.125463 11703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:01.126559 11703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-0/data/instance:
uuid: "53c7b1abbd9b4b3eb89d2993c53cb754"
format_stamp: "Formatted at 2026-09-29 11:03:01 on dist-test-slave-2lzq"
server_key: "ef2bbbfec0db43f8670e149abebcb5b1"
server_key_iv: "3dc37412bae25eb39b5d17fcbf512506"
server_key_version: "encryptionkey@0"
I20260929 11:03:01.126864 11703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance:
uuid: "53c7b1abbd9b4b3eb89d2993c53cb754"
format_stamp: "Formatted at 2026-09-29 11:03:01 on dist-test-slave-2lzq"
server_key: "ef2bbbfec0db43f8670e149abebcb5b1"
server_key_iv: "3dc37412bae25eb39b5d17fcbf512506"
server_key_version: "encryptionkey@0"
I20260929 11:03:01.128196 11703 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 11:03:01.128908 11718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:01.129104 11703 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 11:03:01.129189 11703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-0/wal
uuid: "53c7b1abbd9b4b3eb89d2993c53cb754"
format_stamp: "Formatted at 2026-09-29 11:03:01 on dist-test-slave-2lzq"
server_key: "ef2bbbfec0db43f8670e149abebcb5b1"
server_key_iv: "3dc37412bae25eb39b5d17fcbf512506"
server_key_version: "encryptionkey@0"
I20260929 11:03:01.129240 11703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:01.147929 11703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:01.150000 11703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:01.150190 11703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:01.150408 11703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:03:01.150735 11703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:03:01.150769 11703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:01.150791 11703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:03:01.150841 11703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:01.156392 11703 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.193:37591
I20260929 11:03:01.156435 11831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.193:37591 every 8 connection(s)
I20260929 11:03:01.156755 11703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
Sep 29 11:03:01 dist-test-slave-2lzq krb5kdc[11596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679781, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:03:01.165712 11699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:01.157479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:39447 (local address 127.26.120.254:46241)
0929 11:03:01.157542 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:01.157544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:01.157759 (+ 215us) server_negotiation.cc:407] Connection header received
0929 11:03:01.158313 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:01.158314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:01.158324 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:01.158378 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:01.159306 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.159802 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.160786 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.160986 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.161036 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:01.163099 (+ 2063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:01.163100 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:01.163101 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:01.163107 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:01.164538 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:01.164909 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:01.164910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:01.164910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:01.164960 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:01.165173 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:01.165174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:01.165174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:01.165271 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:01.165321 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:01.165570 (+ 249us) server_negotiation.cc:299] Negotiation successful
0929 11:03:01.165645 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 11:03:01.165930 11834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:01.157480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:46241 (local address 127.26.120.193:39447)
0929 11:03:01.157626 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:01.157662 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:01.158216 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:01.158433 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:01.158436 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:01.159018 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:03:01.159220 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:01.159224 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.159885 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.159886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:01.160707 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:01.160709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.161056 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.161057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:01.161154 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:01.161604 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:03:01.161612 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:03:01.163004 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:03:01.164608 (+ 1604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:01.164614 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:01.164616 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:01.164832 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:01.165018 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:01.165019 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:01.165019 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:01.165119 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:01.165326 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:03:01.165331 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:03:01.165459 (+ 128us) client_negotiation.cc:769] Sending connection context
0929 11:03:01.165542 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 11:03:01.165660 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":79,"threads_started":1}
I20260929 11:03:01.166630 11832 heartbeater.cc:347] Connected to a master server at 127.26.120.254:46241
I20260929 11:03:01.166744 11832 heartbeater.cc:464] Registering TS with master...
I20260929 11:03:01.166785 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 11703
I20260929 11:03:01.166887 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance
I20260929 11:03:01.166970 11832 heartbeater.cc:511] Master 127.26.120.254:46241 requested a full tablet report, sending...
I20260929 11:03:01.167100 27107 external_mini_cluster.cc:1428] Setting key c50191d4eaf169d24d243eb094969f9b
I20260929 11:03:01.167552 11641 ts_manager.cc:194] Registered new tserver with Master: 53c7b1abbd9b4b3eb89d2993c53cb754 (127.26.120.193:37591)
I20260929 11:03:01.168154 11641 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.193@KRBTEST.COM'} at 127.26.120.193:39447
WARNING: no policy specified for kudu/127.26.120.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab.
Entry for principal kudu/127.26.120.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab.
I20260929 11:03:01.190912 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.194:0
--local_ip_for_outbound_sockets=127.26.120.194
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:46241
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:42317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:03:01.270229 11837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:01.270421 11837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:01.270443 11837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:01.271588 11837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:01.271629 11837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:01.271656 11837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.194
I20260929 11:03:01.273087 11837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:42317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:46241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.194
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:01.273332 11837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:01.273561 11837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:01.278493 11843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:01.278575 11837 server_base.cc:1034] running on GCE node
W20260929 11:03:01.278503 11842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:01.278519 11845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:01.278960 11837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:01.279194 11837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:01.280359 11837 hybrid_clock.cc:648] HybridClock initialized: now 1790679781280345 us; error 39 us; skew 500 ppm
Sep 29 11:03:01 dist-test-slave-2lzq krb5kdc[11596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679781, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:03:01.283219 11837 init.cc:377] Logged in from keytab as kudu/127.26.120.194@KRBTEST.COM (short username kudu)
I20260929 11:03:01.283588 11837 webserver.cc:460] Webserver started at http://127.26.120.194:33793/ using document root <none> and password file <none>
I20260929 11:03:01.283773 11837 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:01.283813 11837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:01.283902 11837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:01.284894 11837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-1/data/instance:
uuid: "a2f281272e5944c0ab19c0eef4ce79f5"
format_stamp: "Formatted at 2026-09-29 11:03:01 on dist-test-slave-2lzq"
server_key: "a34940d8a3f85af3c7a487a148ad6931"
server_key_iv: "bc8e2899a408a33bd5b4c36c12e86d3e"
server_key_version: "encryptionkey@0"
I20260929 11:03:01.285207 11837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance:
uuid: "a2f281272e5944c0ab19c0eef4ce79f5"
format_stamp: "Formatted at 2026-09-29 11:03:01 on dist-test-slave-2lzq"
server_key: "a34940d8a3f85af3c7a487a148ad6931"
server_key_iv: "bc8e2899a408a33bd5b4c36c12e86d3e"
server_key_version: "encryptionkey@0"
I20260929 11:03:01.286609 11837 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 11:03:01.287256 11852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:01.287423 11837 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 11:03:01.287528 11837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-1/wal
uuid: "a2f281272e5944c0ab19c0eef4ce79f5"
format_stamp: "Formatted at 2026-09-29 11:03:01 on dist-test-slave-2lzq"
server_key: "a34940d8a3f85af3c7a487a148ad6931"
server_key_iv: "bc8e2899a408a33bd5b4c36c12e86d3e"
server_key_version: "encryptionkey@0"
I20260929 11:03:01.287599 11837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:01.295981 11837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:01.297737 11837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:01.297890 11837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:01.298144 11837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:03:01.298448 11837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:03:01.298481 11837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:01.298502 11837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:03:01.298547 11837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:01.304085 11837 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.194:43093
I20260929 11:03:01.304162 11965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.194:43093 every 8 connection(s)
I20260929 11:03:01.304462 11837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
I20260929 11:03:01.305264 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 11837
I20260929 11:03:01.305359 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance
I20260929 11:03:01.305591 27107 external_mini_cluster.cc:1428] Setting key 89636af289d270d9ed8ead8b6287431b
Sep 29 11:03:01 dist-test-slave-2lzq krb5kdc[11596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679781, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.120.195@KRBTEST.COM; defaulting to no policy
I20260929 11:03:01.315088 11699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:01.305223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:44383 (local address 127.26.120.254:46241)
0929 11:03:01.305293 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:01.305296 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:01.305544 (+ 248us) server_negotiation.cc:407] Connection header received
0929 11:03:01.306257 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:01.306258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:01.306274 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:01.306338 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:01.307688 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.308284 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.309326 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.309539 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.309626 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:01.311793 (+ 2167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:01.311795 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:01.311795 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:01.311805 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:01.313615 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:01.314010 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:01.314011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:01.314011 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:01.314092 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:01.314433 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:01.314434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:01.314434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:01.314590 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:01.314668 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:01.314950 (+ 282us) server_negotiation.cc:299] Negotiation successful
0929 11:03:01.315019 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 11:03:01.315336 11968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:01.305175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:46241 (local address 127.26.120.194:44383)
0929 11:03:01.305396 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:01.305446 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:01.306127 (+ 681us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:01.306442 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:01.306447 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:01.307276 (+ 829us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:03:01.307564 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:01.307570 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.308406 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.308407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:01.309228 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:01.309231 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.309636 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.309637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:01.309767 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:01.310359 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:03:01.310367 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:03:01.311693 (+ 1326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:03:01.313708 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:01.313714 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:01.313716 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:01.313925 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:01.314169 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:01.314170 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:01.314170 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:01.314358 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:01.314680 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:03:01.314686 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:03:01.314823 (+ 137us) client_negotiation.cc:769] Sending connection context
0929 11:03:01.314921 (+ 98us) client_negotiation.cc:240] Negotiation successful
0929 11:03:01.315054 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":76,"threads_started":1}
I20260929 11:03:01.315809 11966 heartbeater.cc:347] Connected to a master server at 127.26.120.254:46241
I20260929 11:03:01.315924 11966 heartbeater.cc:464] Registering TS with master...
I20260929 11:03:01.316074 11966 heartbeater.cc:511] Master 127.26.120.254:46241 requested a full tablet report, sending...
I20260929 11:03:01.316524 11641 ts_manager.cc:194] Registered new tserver with Master: a2f281272e5944c0ab19c0eef4ce79f5 (127.26.120.194:43093)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.195@KRBTEST.COM" created.
I20260929 11:03:01.316952 11641 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.194@KRBTEST.COM'} at 127.26.120.194:44383
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab.
Entry for principal kudu/127.26.120.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab.
I20260929 11:03:01.333009 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.195:0
--local_ip_for_outbound_sockets=127.26.120.195
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:46241
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:42317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:03:01.414259 11971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:01.414467 11971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:01.414492 11971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:01.415576 11971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:01.415611 11971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:01.415637 11971 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.195
I20260929 11:03:01.416998 11971 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:42317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:46241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.195
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:01.417198 11971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:01.417430 11971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:01.422399 11976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:01.422452 11971 server_base.cc:1034] running on GCE node
W20260929 11:03:01.422410 11977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:01.422431 11979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:01.422788 11971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:01.423025 11971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:01.424182 11971 hybrid_clock.cc:648] HybridClock initialized: now 1790679781424175 us; error 28 us; skew 500 ppm
Sep 29 11:03:01 dist-test-slave-2lzq krb5kdc[11596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679781, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:03:01.426575 11971 init.cc:377] Logged in from keytab as kudu/127.26.120.195@KRBTEST.COM (short username kudu)
I20260929 11:03:01.426999 11971 webserver.cc:460] Webserver started at http://127.26.120.195:38107/ using document root <none> and password file <none>
I20260929 11:03:01.427215 11971 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:01.427277 11971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:01.427408 11971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:01.428426 11971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-2/data/instance:
uuid: "d88100b42ea74282af958179e0330e8c"
format_stamp: "Formatted at 2026-09-29 11:03:01 on dist-test-slave-2lzq"
server_key: "495b05fae736beb3d5f631e31bda9768"
server_key_iv: "1336ac2addfb5ad7d6b6c32b2ce2c116"
server_key_version: "encryptionkey@0"
I20260929 11:03:01.428788 11971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance:
uuid: "d88100b42ea74282af958179e0330e8c"
format_stamp: "Formatted at 2026-09-29 11:03:01 on dist-test-slave-2lzq"
server_key: "495b05fae736beb3d5f631e31bda9768"
server_key_iv: "1336ac2addfb5ad7d6b6c32b2ce2c116"
server_key_version: "encryptionkey@0"
I20260929 11:03:01.430083 11971 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 11:03:01.430842 11986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:01.431062 11971 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:03:01.431195 11971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-2/wal
uuid: "d88100b42ea74282af958179e0330e8c"
format_stamp: "Formatted at 2026-09-29 11:03:01 on dist-test-slave-2lzq"
server_key: "495b05fae736beb3d5f631e31bda9768"
server_key_iv: "1336ac2addfb5ad7d6b6c32b2ce2c116"
server_key_version: "encryptionkey@0"
I20260929 11:03:01.431272 11971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:01.450920 11971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:01.453094 11971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:01.453258 11971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:01.453466 11971 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:03:01.453802 11971 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:03:01.453838 11971 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:01.453862 11971 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:03:01.453912 11971 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:01.459709 11971 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.195:42983
I20260929 11:03:01.459789 12099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.195:42983 every 8 connection(s)
I20260929 11:03:01.460108 11971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
Sep 29 11:03:01 dist-test-slave-2lzq krb5kdc[11596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679781, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:03:01.468259 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 11971
I20260929 11:03:01.468375 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance
I20260929 11:03:01.468576 27107 external_mini_cluster.cc:1428] Setting key 63712fd0cd1c9499ffdc1bc931f0bd42
I20260929 11:03:01.469748 11699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:01.460817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:40219 (local address 127.26.120.254:46241)
0929 11:03:01.460887 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:01.460889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:01.461087 (+ 198us) server_negotiation.cc:407] Connection header received
0929 11:03:01.461543 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:01.461544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:01.461555 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:01.461605 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:01.462594 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.463120 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.464114 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.464297 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.464348 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:01.466717 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:01.466719 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:01.466719 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:01.466726 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:01.468226 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:01.468694 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:01.468696 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:01.468696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:01.468760 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:01.469054 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:01.469055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:01.469056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:01.469183 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:01.469302 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:01.469576 (+ 274us) server_negotiation.cc:299] Negotiation successful
0929 11:03:01.469655 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 11:03:01.469966 12102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:01.460814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:46241 (local address 127.26.120.195:40219)
0929 11:03:01.460962 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:01.460997 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:01.461460 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:01.461658 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:01.461662 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:01.462306 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:03:01.462511 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:01.462516 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.463219 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.463220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:01.464025 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:01.464027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.464385 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.464392 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:01.464517 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:01.465086 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:03:01.465094 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:03:01.466625 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:03:01.468317 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:01.468346 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:01.468348 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:01.468624 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:01.468840 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:01.468841 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:01.468841 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:01.468982 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:01.469289 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:03:01.469296 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:03:01.469441 (+ 145us) client_negotiation.cc:769] Sending connection context
0929 11:03:01.469553 (+ 112us) client_negotiation.cc:240] Negotiation successful
0929 11:03:01.469727 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":86,"threads_started":1}
I20260929 11:03:01.470454 12100 heartbeater.cc:347] Connected to a master server at 127.26.120.254:46241
I20260929 11:03:01.470561 12100 heartbeater.cc:464] Registering TS with master...
I20260929 11:03:01.470742 12100 heartbeater.cc:511] Master 127.26.120.254:46241 requested a full tablet report, sending...
I20260929 11:03:01.471097 11641 ts_manager.cc:194] Registered new tserver with Master: d88100b42ea74282af958179e0330e8c (127.26.120.195:42983)
I20260929 11:03:01.471439 11641 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.195@KRBTEST.COM'} at 127.26.120.195:40219
I20260929 11:03:01.480535 27107 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 11:03:01.488755 11699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:01.482124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33928 (local address 127.26.120.254:46241)
0929 11:03:01.482172 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:01.482174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:01.482337 (+ 163us) server_negotiation.cc:407] Connection header received
0929 11:03:01.482436 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:01.482437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:01.482447 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:01.482506 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:01.483116 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.483631 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.484452 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.484662 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.484756 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:01.485989 (+ 1233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:01.485991 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:01.485992 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:01.485999 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:01.487503 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:01.487892 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:01.487894 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:01.487894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:01.487958 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:01.488165 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:01.488166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:01.488167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:01.488320 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:01.488429 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:01.488560 (+ 131us) server_negotiation.cc:299] Negotiation successful
0929 11:03:01.488639 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 11:03:01.490302 11641 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33928:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 11:03:01.492004 11641 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 11:03:01.498157 12108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:01.494148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:42983 (local address 127.0.0.1:60182)
0929 11:03:01.494436 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:01.494448 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:01.494504 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:01.494765 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:01.494768 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:01.494769 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:03:01.494841 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:01.494842 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.495814 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.495815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:01.496843 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:01.496845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.497787 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.497788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:01.497884 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:01.497905 (+ 21us) client_negotiation.cc:769] Sending connection context
0929 11:03:01.497995 (+ 90us) client_negotiation.cc:240] Negotiation successful
0929 11:03:01.498081 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":2,"thread_start_us":55,"threads_started":1}
I20260929 11:03:01.498171 12112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:01.494408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60182 (local address 127.26.120.195:42983)
0929 11:03:01.494587 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:01.494592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:01.494601 (+ 9us) server_negotiation.cc:407] Connection header received
0929 11:03:01.494639 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:01.494646 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:01.494658 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:01.494758 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:03:01.494943 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.495708 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.496944 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.497688 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.497789 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:01.497852 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:01.498015 (+ 163us) server_negotiation.cc:299] Negotiation successful
0929 11:03:01.498080 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":86,"threads_started":1}
I20260929 11:03:01.498764 12111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:01.494233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34190 (local address 127.26.120.193:37591)
0929 11:03:01.494803 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:01.494808 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:01.494818 (+ 10us) server_negotiation.cc:407] Connection header received
0929 11:03:01.494855 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:01.494857 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:01.494873 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:01.494944 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:03:01.495129 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.496280 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.497640 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.498371 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.498456 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:01.498516 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:01.498615 (+ 99us) server_negotiation.cc:299] Negotiation successful
0929 11:03:01.498663 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":116,"threads_started":1}
I20260929 11:03:01.498790 12110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:01.494148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:43093 (local address 127.0.0.1:58356)
0929 11:03:01.494394 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:01.494436 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:01.494488 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:01.495014 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:01.495015 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:01.495017 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:03:01.495074 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:01.495076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.496424 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.496426 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:01.497456 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:01.497459 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.498523 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.498524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:01.498600 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:01.498605 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 11:03:01.498636 (+ 31us) client_negotiation.cc:240] Negotiation successful
0929 11:03:01.498675 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":102,"threads_started":1}
I20260929 11:03:01.498741 12109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:01.494154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:37591 (local address 127.0.0.1:34190)
0929 11:03:01.494600 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:01.494609 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:01.494648 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:01.495024 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:01.495025 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:01.495027 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:03:01.495064 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:01.495065 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.496400 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.496401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:01.497553 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:01.497556 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.498460 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.498462 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:01.498530 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:01.498534 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 11:03:01.498600 (+ 66us) client_negotiation.cc:240] Negotiation successful
0929 11:03:01.498644 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"mutex_wait_us":29,"thread_start_us":57,"threads_started":1}
I20260929 11:03:01.499104 12034 tablet_service.cc:1467] Processing CreateTablet for tablet fba7d4035e9042eca886ee379997f735 (DEFAULT_TABLE table=test-table [id=c300676afbe14a519465a0f5d22ca3e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:03:01.499284 12113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:01.494601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58356 (local address 127.26.120.194:43093)
0929 11:03:01.494848 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:01.494852 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:01.494862 (+ 10us) server_negotiation.cc:407] Connection header received
0929 11:03:01.494901 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:01.494903 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:01.494919 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:01.495152 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:03:01.495236 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.496297 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.497550 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.498433 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.498792 (+ 359us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:01.499122 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:01.499148 (+ 26us) server_negotiation.cc:299] Negotiation successful
0929 11:03:01.499205 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":84,"threads_started":1}
I20260929 11:03:01.499483 12034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fba7d4035e9042eca886ee379997f735. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:01.499580 11766 tablet_service.cc:1467] Processing CreateTablet for tablet fba7d4035e9042eca886ee379997f735 (DEFAULT_TABLE table=test-table [id=c300676afbe14a519465a0f5d22ca3e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:03:01.499831 11766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fba7d4035e9042eca886ee379997f735. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:01.502005 12114 tablet_bootstrap.cc:492] T fba7d4035e9042eca886ee379997f735 P d88100b42ea74282af958179e0330e8c: Bootstrap starting.
I20260929 11:03:01.502144 12115 tablet_bootstrap.cc:492] T fba7d4035e9042eca886ee379997f735 P 53c7b1abbd9b4b3eb89d2993c53cb754: Bootstrap starting.
I20260929 11:03:01.502656 11900 tablet_service.cc:1467] Processing CreateTablet for tablet fba7d4035e9042eca886ee379997f735 (DEFAULT_TABLE table=test-table [id=c300676afbe14a519465a0f5d22ca3e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:03:01.502765 12114 tablet_bootstrap.cc:654] T fba7d4035e9042eca886ee379997f735 P d88100b42ea74282af958179e0330e8c: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:01.502774 12115 tablet_bootstrap.cc:654] T fba7d4035e9042eca886ee379997f735 P 53c7b1abbd9b4b3eb89d2993c53cb754: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:01.502978 11900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fba7d4035e9042eca886ee379997f735. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:01.503185 12115 log.cc:824] T fba7d4035e9042eca886ee379997f735 P 53c7b1abbd9b4b3eb89d2993c53cb754: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:01.503400 12114 log.cc:824] T fba7d4035e9042eca886ee379997f735 P d88100b42ea74282af958179e0330e8c: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:01.503978 12115 tablet_bootstrap.cc:492] T fba7d4035e9042eca886ee379997f735 P 53c7b1abbd9b4b3eb89d2993c53cb754: No bootstrap required, opened a new log
I20260929 11:03:01.504051 12115 ts_tablet_manager.cc:1397] T fba7d4035e9042eca886ee379997f735 P 53c7b1abbd9b4b3eb89d2993c53cb754: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 11:03:01.506191 12115 raft_consensus.cc:348] T fba7d4035e9042eca886ee379997f735 P 53c7b1abbd9b4b3eb89d2993c53cb754 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53c7b1abbd9b4b3eb89d2993c53cb754" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37591 } } peers { permanent_uuid: "d88100b42ea74282af958179e0330e8c" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 42983 } } peers { permanent_uuid: "a2f281272e5944c0ab19c0eef4ce79f5" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 43093 } }
I20260929 11:03:01.506373 12115 raft_consensus.cc:374] T fba7d4035e9042eca886ee379997f735 P 53c7b1abbd9b4b3eb89d2993c53cb754 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:01.506428 12115 raft_consensus.cc:729] T fba7d4035e9042eca886ee379997f735 P 53c7b1abbd9b4b3eb89d2993c53cb754 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53c7b1abbd9b4b3eb89d2993c53cb754, State: Initialized, Role: FOLLOWER
I20260929 11:03:01.506374 12118 tablet_bootstrap.cc:492] T fba7d4035e9042eca886ee379997f735 P a2f281272e5944c0ab19c0eef4ce79f5: Bootstrap starting.
I20260929 11:03:01.506582 12114 tablet_bootstrap.cc:492] T fba7d4035e9042eca886ee379997f735 P d88100b42ea74282af958179e0330e8c: No bootstrap required, opened a new log
I20260929 11:03:01.506556 12115 consensus_queue.cc:260] T fba7d4035e9042eca886ee379997f735 P 53c7b1abbd9b4b3eb89d2993c53cb754 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53c7b1abbd9b4b3eb89d2993c53cb754" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37591 } } peers { permanent_uuid: "d88100b42ea74282af958179e0330e8c" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 42983 } } peers { permanent_uuid: "a2f281272e5944c0ab19c0eef4ce79f5" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 43093 } }
I20260929 11:03:01.506646 12114 ts_tablet_manager.cc:1397] T fba7d4035e9042eca886ee379997f735 P d88100b42ea74282af958179e0330e8c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260929 11:03:01.506845 12115 ts_tablet_manager.cc:1428] T fba7d4035e9042eca886ee379997f735 P 53c7b1abbd9b4b3eb89d2993c53cb754: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 11:03:01.506851 11832 heartbeater.cc:503] Master 127.26.120.254:46241 was elected leader, sending a full tablet report...
I20260929 11:03:01.507175 12118 tablet_bootstrap.cc:654] T fba7d4035e9042eca886ee379997f735 P a2f281272e5944c0ab19c0eef4ce79f5: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:01.507781 12118 log.cc:824] T fba7d4035e9042eca886ee379997f735 P a2f281272e5944c0ab19c0eef4ce79f5: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:01.508548 12118 tablet_bootstrap.cc:492] T fba7d4035e9042eca886ee379997f735 P a2f281272e5944c0ab19c0eef4ce79f5: No bootstrap required, opened a new log
I20260929 11:03:01.508620 12118 ts_tablet_manager.cc:1397] T fba7d4035e9042eca886ee379997f735 P a2f281272e5944c0ab19c0eef4ce79f5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 11:03:01.508852 12114 raft_consensus.cc:348] T fba7d4035e9042eca886ee379997f735 P d88100b42ea74282af958179e0330e8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53c7b1abbd9b4b3eb89d2993c53cb754" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37591 } } peers { permanent_uuid: "d88100b42ea74282af958179e0330e8c" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 42983 } } peers { permanent_uuid: "a2f281272e5944c0ab19c0eef4ce79f5" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 43093 } }
I20260929 11:03:01.509020 12114 raft_consensus.cc:374] T fba7d4035e9042eca886ee379997f735 P d88100b42ea74282af958179e0330e8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:01.509060 12114 raft_consensus.cc:729] T fba7d4035e9042eca886ee379997f735 P d88100b42ea74282af958179e0330e8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d88100b42ea74282af958179e0330e8c, State: Initialized, Role: FOLLOWER
I20260929 11:03:01.509169 12114 consensus_queue.cc:260] T fba7d4035e9042eca886ee379997f735 P d88100b42ea74282af958179e0330e8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53c7b1abbd9b4b3eb89d2993c53cb754" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37591 } } peers { permanent_uuid: "d88100b42ea74282af958179e0330e8c" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 42983 } } peers { permanent_uuid: "a2f281272e5944c0ab19c0eef4ce79f5" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 43093 } }
I20260929 11:03:01.509431 12114 ts_tablet_manager.cc:1428] T fba7d4035e9042eca886ee379997f735 P d88100b42ea74282af958179e0330e8c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 11:03:01.509467 12100 heartbeater.cc:503] Master 127.26.120.254:46241 was elected leader, sending a full tablet report...
I20260929 11:03:01.510608 12118 raft_consensus.cc:348] T fba7d4035e9042eca886ee379997f735 P a2f281272e5944c0ab19c0eef4ce79f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53c7b1abbd9b4b3eb89d2993c53cb754" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37591 } } peers { permanent_uuid: "d88100b42ea74282af958179e0330e8c" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 42983 } } peers { permanent_uuid: "a2f281272e5944c0ab19c0eef4ce79f5" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 43093 } }
I20260929 11:03:01.510738 12118 raft_consensus.cc:374] T fba7d4035e9042eca886ee379997f735 P a2f281272e5944c0ab19c0eef4ce79f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:01.510761 12118 raft_consensus.cc:729] T fba7d4035e9042eca886ee379997f735 P a2f281272e5944c0ab19c0eef4ce79f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2f281272e5944c0ab19c0eef4ce79f5, State: Initialized, Role: FOLLOWER
I20260929 11:03:01.510861 12118 consensus_queue.cc:260] T fba7d4035e9042eca886ee379997f735 P a2f281272e5944c0ab19c0eef4ce79f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53c7b1abbd9b4b3eb89d2993c53cb754" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37591 } } peers { permanent_uuid: "d88100b42ea74282af958179e0330e8c" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 42983 } } peers { permanent_uuid: "a2f281272e5944c0ab19c0eef4ce79f5" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 43093 } }
I20260929 11:03:01.511082 12118 ts_tablet_manager.cc:1428] T fba7d4035e9042eca886ee379997f735 P a2f281272e5944c0ab19c0eef4ce79f5: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 11:03:01.511139 11966 heartbeater.cc:503] Master 127.26.120.254:46241 was elected leader, sending a full tablet report...
I20260929 11:03:01.548244 12122 raft_consensus.cc:482] T fba7d4035e9042eca886ee379997f735 P a2f281272e5944c0ab19c0eef4ce79f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 11:03:01.548398 12122 raft_consensus.cc:504] T fba7d4035e9042eca886ee379997f735 P a2f281272e5944c0ab19c0eef4ce79f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53c7b1abbd9b4b3eb89d2993c53cb754" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37591 } } peers { permanent_uuid: "d88100b42ea74282af958179e0330e8c" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 42983 } } peers { permanent_uuid: "a2f281272e5944c0ab19c0eef4ce79f5" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 43093 } }
I20260929 11:03:01.548851 12122 leader_election.cc:288] T fba7d4035e9042eca886ee379997f735 P a2f281272e5944c0ab19c0eef4ce79f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 53c7b1abbd9b4b3eb89d2993c53cb754 (127.26.120.193:37591), d88100b42ea74282af958179e0330e8c (127.26.120.195:42983)
I20260929 11:03:01.550148 12119 raft_consensus.cc:482] T fba7d4035e9042eca886ee379997f735 P 53c7b1abbd9b4b3eb89d2993c53cb754 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 11:03:01.550266 12119 raft_consensus.cc:504] T fba7d4035e9042eca886ee379997f735 P 53c7b1abbd9b4b3eb89d2993c53cb754 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53c7b1abbd9b4b3eb89d2993c53cb754" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37591 } } peers { permanent_uuid: "d88100b42ea74282af958179e0330e8c" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 42983 } } peers { permanent_uuid: "a2f281272e5944c0ab19c0eef4ce79f5" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 43093 } }
I20260929 11:03:01.550624 12119 leader_election.cc:288] T fba7d4035e9042eca886ee379997f735 P 53c7b1abbd9b4b3eb89d2993c53cb754 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d88100b42ea74282af958179e0330e8c (127.26.120.195:42983), a2f281272e5944c0ab19c0eef4ce79f5 (127.26.120.194:43093)
I20260929 11:03:01.552376 11968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:01.549045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:42983 (local address 127.26.120.194:52775)
0929 11:03:01.549113 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:01.549126 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:01.549175 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:01.549387 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:01.549388 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:01.549390 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:03:01.549460 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:01.549462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.550166 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.550167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:01.551319 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:01.551322 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.552111 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.552112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:01.552191 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:01.552195 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 11:03:01.552237 (+ 42us) client_negotiation.cc:240] Negotiation successful
0929 11:03:01.552275 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260929 11:03:01.552659 12111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:01.549177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:46033 (local address 127.26.120.193:37591)
0929 11:03:01.549235 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:01.549237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:01.549431 (+ 194us) server_negotiation.cc:407] Connection header received
0929 11:03:01.549502 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:01.549503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:01.549512 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:01.549580 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:03:01.549745 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.550366 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.551742 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.552293 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.552337 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:01.552344 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:01.552557 (+ 213us) server_negotiation.cc:299] Negotiation successful
0929 11:03:01.552594 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260929 11:03:01.552667 12112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:01.549166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:52775 (local address 127.26.120.195:42983)
0929 11:03:01.549213 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:01.549215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:01.549224 (+ 9us) server_negotiation.cc:407] Connection header received
0929 11:03:01.549252 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:01.549253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:01.549269 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:01.549356 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:03:01.549546 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.550026 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.551408 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.552005 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.552071 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:01.552080 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:01.552559 (+ 479us) server_negotiation.cc:299] Negotiation successful
0929 11:03:01.552602 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 11:03:01.552899 12123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:01.549059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:37591 (local address 127.26.120.194:46033)
0929 11:03:01.549355 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:01.549363 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:01.549445 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:01.549595 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:01.549596 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:01.549598 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:03:01.549676 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:01.549678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.550462 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.550463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:01.551652 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:01.551654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.552402 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.552402 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:01.552483 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:01.552486 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 11:03:01.552786 (+ 300us) client_negotiation.cc:240] Negotiation successful
0929 11:03:01.552825 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":74,"thread_start_us":124,"threads_started":1}
I20260929 11:03:01.552938 12054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fba7d4035e9042eca886ee379997f735" candidate_uuid: "a2f281272e5944c0ab19c0eef4ce79f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d88100b42ea74282af958179e0330e8c" is_pre_election: true
I20260929 11:03:01.553092 12054 raft_consensus.cc:2454] T fba7d4035e9042eca886ee379997f735 P d88100b42ea74282af958179e0330e8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2f281272e5944c0ab19c0eef4ce79f5 in term 0.
I20260929 11:03:01.553164 11786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fba7d4035e9042eca886ee379997f735" candidate_uuid: "a2f281272e5944c0ab19c0eef4ce79f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53c7b1abbd9b4b3eb89d2993c53cb754" is_pre_election: true
I20260929 11:03:01.553272 11786 raft_consensus.cc:2454] T fba7d4035e9042eca886ee379997f735 P 53c7b1abbd9b4b3eb89d2993c53cb754 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2f281272e5944c0ab19c0eef4ce79f5 in term 0.
I20260929 11:03:01.553324 11854 leader_election.cc:302] T fba7d4035e9042eca886ee379997f735 P a2f281272e5944c0ab19c0eef4ce79f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2f281272e5944c0ab19c0eef4ce79f5, d88100b42ea74282af958179e0330e8c; no voters:
I20260929 11:03:01.553488 12122 raft_consensus.cc:2781] T fba7d4035e9042eca886ee379997f735 P a2f281272e5944c0ab19c0eef4ce79f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 11:03:01.553571 12122 raft_consensus.cc:482] T fba7d4035e9042eca886ee379997f735 P a2f281272e5944c0ab19c0eef4ce79f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 11:03:01.553596 12122 raft_consensus.cc:3037] T fba7d4035e9042eca886ee379997f735 P a2f281272e5944c0ab19c0eef4ce79f5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:01.553992 12113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:01.550966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:39571 (local address 127.26.120.194:43093)
0929 11:03:01.551001 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:01.551003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:01.551009 (+ 6us) server_negotiation.cc:407] Connection header received
0929 11:03:01.551027 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:01.551028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:01.551035 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:01.551076 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:03:01.551222 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.551872 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.553138 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.553671 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.553741 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:01.553748 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:01.553905 (+ 157us) server_negotiation.cc:299] Negotiation successful
0929 11:03:01.553941 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 11:03:01.553996 11834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:01.550753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:43093 (local address 127.26.120.193:39571)
0929 11:03:01.550848 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:01.550858 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:01.550898 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:01.551107 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:01.551108 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:01.551109 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:03:01.551160 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:01.551161 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.551952 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.551953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:01.553040 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:01.553042 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.553740 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.553741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:01.553821 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:01.553825 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 11:03:01.553893 (+ 68us) client_negotiation.cc:240] Negotiation successful
0929 11:03:01.553928 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260929 11:03:01.554515 11920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fba7d4035e9042eca886ee379997f735" candidate_uuid: "53c7b1abbd9b4b3eb89d2993c53cb754" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2f281272e5944c0ab19c0eef4ce79f5" is_pre_election: true
I20260929 11:03:01.554628 12122 raft_consensus.cc:504] T fba7d4035e9042eca886ee379997f735 P a2f281272e5944c0ab19c0eef4ce79f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53c7b1abbd9b4b3eb89d2993c53cb754" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37591 } } peers { permanent_uuid: "d88100b42ea74282af958179e0330e8c" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 42983 } } peers { permanent_uuid: "a2f281272e5944c0ab19c0eef4ce79f5" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 43093 } }
I20260929 11:03:01.554796 12122 leader_election.cc:288] T fba7d4035e9042eca886ee379997f735 P a2f281272e5944c0ab19c0eef4ce79f5 [CANDIDATE]: Term 1 election: Requested vote from peers 53c7b1abbd9b4b3eb89d2993c53cb754 (127.26.120.193:37591), d88100b42ea74282af958179e0330e8c (127.26.120.195:42983)
I20260929 11:03:01.554908 11786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fba7d4035e9042eca886ee379997f735" candidate_uuid: "a2f281272e5944c0ab19c0eef4ce79f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53c7b1abbd9b4b3eb89d2993c53cb754"
I20260929 11:03:01.554958 11920 raft_consensus.cc:2379] T fba7d4035e9042eca886ee379997f735 P a2f281272e5944c0ab19c0eef4ce79f5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53c7b1abbd9b4b3eb89d2993c53cb754 in current term 1: Already voted for candidate a2f281272e5944c0ab19c0eef4ce79f5 in this term.
I20260929 11:03:01.554975 11786 raft_consensus.cc:3037] T fba7d4035e9042eca886ee379997f735 P 53c7b1abbd9b4b3eb89d2993c53cb754 [term 0 FOLLOWER]: Advancing to term 1
W20260929 11:03:01.555218 11967 tablet.cc:2369] T fba7d4035e9042eca886ee379997f735 P a2f281272e5944c0ab19c0eef4ce79f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 11:03:01.555238 12054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fba7d4035e9042eca886ee379997f735" candidate_uuid: "a2f281272e5944c0ab19c0eef4ce79f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d88100b42ea74282af958179e0330e8c"
I20260929 11:03:01.555305 12054 raft_consensus.cc:3037] T fba7d4035e9042eca886ee379997f735 P d88100b42ea74282af958179e0330e8c [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:01.555768 12124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:01.550807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:42983 (local address 127.26.120.193:60931)
0929 11:03:01.551215 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:01.551224 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:01.551297 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:01.551499 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:01.551500 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:01.551502 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:03:01.551560 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:01.551561 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.552401 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.552402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:01.554402 (+ 2000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:01.554405 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.555524 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.555525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:01.555592 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:01.555596 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 11:03:01.555653 (+ 57us) client_negotiation.cc:240] Negotiation successful
0929 11:03:01.555691 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":88,"threads_started":1}
I20260929 11:03:01.555835 12125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:01.550976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:60931 (local address 127.26.120.195:42983)
0929 11:03:01.551104 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:01.551105 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:01.551273 (+ 168us) server_negotiation.cc:407] Connection header received
0929 11:03:01.551384 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:01.551385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:01.551395 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:01.551439 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:03:01.551632 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.552298 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.554501 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.555064 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.555112 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:01.555121 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:01.555677 (+ 556us) server_negotiation.cc:299] Negotiation successful
0929 11:03:01.555726 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":77,"threads_started":1}
I20260929 11:03:01.555883 11786 raft_consensus.cc:2454] T fba7d4035e9042eca886ee379997f735 P 53c7b1abbd9b4b3eb89d2993c53cb754 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2f281272e5944c0ab19c0eef4ce79f5 in term 1.
I20260929 11:03:01.556062 11855 leader_election.cc:302] T fba7d4035e9042eca886ee379997f735 P a2f281272e5944c0ab19c0eef4ce79f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53c7b1abbd9b4b3eb89d2993c53cb754, a2f281272e5944c0ab19c0eef4ce79f5; no voters:
I20260929 11:03:01.556159 12122 raft_consensus.cc:2781] T fba7d4035e9042eca886ee379997f735 P a2f281272e5944c0ab19c0eef4ce79f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:03:01.556115 12053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fba7d4035e9042eca886ee379997f735" candidate_uuid: "53c7b1abbd9b4b3eb89d2993c53cb754" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d88100b42ea74282af958179e0330e8c" is_pre_election: true
I20260929 11:03:01.556288 12054 raft_consensus.cc:2454] T fba7d4035e9042eca886ee379997f735 P d88100b42ea74282af958179e0330e8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2f281272e5944c0ab19c0eef4ce79f5 in term 1.
I20260929 11:03:01.556319 12122 raft_consensus.cc:686] T fba7d4035e9042eca886ee379997f735 P a2f281272e5944c0ab19c0eef4ce79f5 [term 1 LEADER]: Becoming Leader. State: Replica: a2f281272e5944c0ab19c0eef4ce79f5, State: Running, Role: LEADER
I20260929 11:03:01.556411 12122 consensus_queue.cc:237] T fba7d4035e9042eca886ee379997f735 P a2f281272e5944c0ab19c0eef4ce79f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53c7b1abbd9b4b3eb89d2993c53cb754" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37591 } } peers { permanent_uuid: "d88100b42ea74282af958179e0330e8c" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 42983 } } peers { permanent_uuid: "a2f281272e5944c0ab19c0eef4ce79f5" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 43093 } }
I20260929 11:03:01.556434 11720 leader_election.cc:302] T fba7d4035e9042eca886ee379997f735 P 53c7b1abbd9b4b3eb89d2993c53cb754 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53c7b1abbd9b4b3eb89d2993c53cb754; no voters: a2f281272e5944c0ab19c0eef4ce79f5, d88100b42ea74282af958179e0330e8c
I20260929 11:03:01.556542 12119 raft_consensus.cc:2726] T fba7d4035e9042eca886ee379997f735 P 53c7b1abbd9b4b3eb89d2993c53cb754 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260929 11:03:01.557240 11641 catalog_manager.cc:5495] T fba7d4035e9042eca886ee379997f735 P a2f281272e5944c0ab19c0eef4ce79f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2f281272e5944c0ab19c0eef4ce79f5 (127.26.120.194). New cstate: current_term: 1 leader_uuid: "a2f281272e5944c0ab19c0eef4ce79f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53c7b1abbd9b4b3eb89d2993c53cb754" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d88100b42ea74282af958179e0330e8c" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 42983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2f281272e5944c0ab19c0eef4ce79f5" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 43093 } health_report { overall_health: HEALTHY } } }
I20260929 11:03:01.575860 12113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:01.573229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58368 (local address 127.26.120.194:43093)
0929 11:03:01.573293 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:01.573295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:01.573305 (+ 10us) server_negotiation.cc:407] Connection header received
0929 11:03:01.573409 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:01.573409 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:01.573437 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:01.573521 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 11:03:01.573661 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.574237 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.575016 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.575222 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.575297 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:01.575483 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 11:03:01.575592 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 11:03:01.575702 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:01.575719 (+ 17us) server_negotiation.cc:299] Negotiation successful
0929 11:03:01.575739 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 11:03:01.577780 12054 raft_consensus.cc:1261] T fba7d4035e9042eca886ee379997f735 P d88100b42ea74282af958179e0330e8c [term 1 FOLLOWER]: Refusing update from remote peer a2f281272e5944c0ab19c0eef4ce79f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 11:03:01.577786 11786 raft_consensus.cc:1261] T fba7d4035e9042eca886ee379997f735 P 53c7b1abbd9b4b3eb89d2993c53cb754 [term 1 FOLLOWER]: Refusing update from remote peer a2f281272e5944c0ab19c0eef4ce79f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 11:03:01.578141 12126 consensus_queue.cc:1035] T fba7d4035e9042eca886ee379997f735 P a2f281272e5944c0ab19c0eef4ce79f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53c7b1abbd9b4b3eb89d2993c53cb754" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 11:03:01.578303 12122 consensus_queue.cc:1035] T fba7d4035e9042eca886ee379997f735 P a2f281272e5944c0ab19c0eef4ce79f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d88100b42ea74282af958179e0330e8c" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 42983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 11:03:01.581334 12130 mvcc.cc:205] Tried to move back new op lower bound from 7334624385339342848 to 7334624385255563264. Current Snapshot: MvccSnapshot[applied={T|T < 7334624385339342848}]
I20260929 11:03:01.582348 12131 mvcc.cc:205] Tried to move back new op lower bound from 7334624385339342848 to 7334624385255563264. Current Snapshot: MvccSnapshot[applied={T|T < 7334624385339342848}]
I20260929 11:03:01.583385 11640 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 11:03:01.583487 11640 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 11:03:01.584386 11640 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 17abfdfaa64c4f08855087a3405369b3: Sending DeleteTablet for 3 replicas of tablet fba7d4035e9042eca886ee379997f735
I20260929 11:03:01.584762 11766 tablet_service.cc:1514] Processing DeleteTablet for tablet fba7d4035e9042eca886ee379997f735 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:03:01 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:34190
I20260929 11:03:01.584897 12034 tablet_service.cc:1514] Processing DeleteTablet for tablet fba7d4035e9042eca886ee379997f735 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:03:01 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:60182
I20260929 11:03:01.584983 11900 tablet_service.cc:1514] Processing DeleteTablet for tablet fba7d4035e9042eca886ee379997f735 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:03:01 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:58356
I20260929 11:03:01.585175 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 11703
I20260929 11:03:01.585194 12138 tablet_replica.cc:330] T fba7d4035e9042eca886ee379997f735 P 53c7b1abbd9b4b3eb89d2993c53cb754: stopping tablet replica
I20260929 11:03:01.586621 12139 tablet_replica.cc:330] T fba7d4035e9042eca886ee379997f735 P d88100b42ea74282af958179e0330e8c: stopping tablet replica
I20260929 11:03:01.586705 12139 raft_consensus.cc:2229] T fba7d4035e9042eca886ee379997f735 P d88100b42ea74282af958179e0330e8c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 11:03:01.586778 12139 raft_consensus.cc:2258] T fba7d4035e9042eca886ee379997f735 P d88100b42ea74282af958179e0330e8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 11:03:01.586992 12140 tablet_replica.cc:330] T fba7d4035e9042eca886ee379997f735 P a2f281272e5944c0ab19c0eef4ce79f5: stopping tablet replica
I20260929 11:03:01.587080 12140 raft_consensus.cc:2229] T fba7d4035e9042eca886ee379997f735 P a2f281272e5944c0ab19c0eef4ce79f5 [term 1 LEADER]: Raft consensus shutting down.
I20260929 11:03:01.587145 12140 raft_consensus.cc:2258] T fba7d4035e9042eca886ee379997f735 P a2f281272e5944c0ab19c0eef4ce79f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 11:03:01.587613 12140 ts_tablet_manager.cc:1905] T fba7d4035e9042eca886ee379997f735 P a2f281272e5944c0ab19c0eef4ce79f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 11:03:01.589097 12140 ts_tablet_manager.cc:1918] T fba7d4035e9042eca886ee379997f735 P a2f281272e5944c0ab19c0eef4ce79f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 11:03:01.589178 12140 log.cc:1196] T fba7d4035e9042eca886ee379997f735 P a2f281272e5944c0ab19c0eef4ce79f5: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-1/wal/wals/fba7d4035e9042eca886ee379997f735
I20260929 11:03:01.589406 12140 ts_tablet_manager.cc:1939] T fba7d4035e9042eca886ee379997f735 P a2f281272e5944c0ab19c0eef4ce79f5: Deleting consensus metadata
I20260929 11:03:01.589799 11625 catalog_manager.cc:4841] TS a2f281272e5944c0ab19c0eef4ce79f5 (127.26.120.194:43093): tablet fba7d4035e9042eca886ee379997f735 (table test-table [id=c300676afbe14a519465a0f5d22ca3e9]) successfully deleted
I20260929 11:03:01.590394 12139 ts_tablet_manager.cc:1905] T fba7d4035e9042eca886ee379997f735 P d88100b42ea74282af958179e0330e8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 11:03:01.591789 12139 ts_tablet_manager.cc:1918] T fba7d4035e9042eca886ee379997f735 P d88100b42ea74282af958179e0330e8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 11:03:01.591879 12139 log.cc:1196] T fba7d4035e9042eca886ee379997f735 P d88100b42ea74282af958179e0330e8c: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790679549986039-27107-0/minicluster-data/ts-2/wal/wals/fba7d4035e9042eca886ee379997f735
I20260929 11:03:01.592384 12139 ts_tablet_manager.cc:1939] T fba7d4035e9042eca886ee379997f735 P d88100b42ea74282af958179e0330e8c: Deleting consensus metadata
I20260929 11:03:01.592788 11626 catalog_manager.cc:4841] TS d88100b42ea74282af958179e0330e8c (127.26.120.195:42983): tablet fba7d4035e9042eca886ee379997f735 (table test-table [id=c300676afbe14a519465a0f5d22ca3e9]) successfully deleted
W20260929 11:03:01.593108 11627 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.120.193:37591 (error 108)
I20260929 11:03:01.593398 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 11837
I20260929 11:03:01.594995 12109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:01.594852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:37591 (local address 127.0.0.1:34192)
0929 11:03:01.594918 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:01.594961 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.193:37591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260929 11:03:01.595130 11627 catalog_manager.cc:4584] TS 53c7b1abbd9b4b3eb89d2993c53cb754 (127.26.120.193:37591): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fba7d4035e9042eca886ee379997f735: Network error: Client connection negotiation failed: client connection to 127.26.120.193:37591: connect: Connection refused (error 111)
I20260929 11:03:01.599020 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 11971
I20260929 11:03:01.603562 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 11609
2026-09-29T11:03:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1958 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-29T11:03:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T11:03:01Z Disabled control of system clock
I20260929 11:03:01.631505 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:35657
--webserver_interface=127.26.120.254
--webserver_port=0
--builtin_ntp_servers=127.26.120.212:44131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:35657
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 11:03:01.711423 12145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:01.711654 12145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:01.711675 12145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:01.712773 12145 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:03:01.712808 12145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:01.712821 12145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:01.712833 12145 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:03:01.712845 12145 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:03:01.714138 12145 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:44131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:35657
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:35657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:01.714342 12145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:01.714587 12145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:01.719590 12151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:01.719615 12145 server_base.cc:1034] running on GCE node
W20260929 11:03:01.719597 12153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:01.719597 12150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:01.720168 12145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:01.720486 12145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:01.721669 12145 hybrid_clock.cc:648] HybridClock initialized: now 1790679781721654 us; error 32 us; skew 500 ppm
I20260929 11:03:01.723120 12145 webserver.cc:460] Webserver started at http://127.26.120.254:39567/ using document root <none> and password file <none>
I20260929 11:03:01.723424 12145 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:01.723476 12145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:01.723634 12145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:01.724750 12145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "15e386b2b0c244168635658ca6299b3e"
format_stamp: "Formatted at 2026-09-29 11:03:01 on dist-test-slave-2lzq"
server_key: "ebf12624c1345453f5aa5e1bb7c773e1"
server_key_iv: "b280426589aaa4cce5d87f8310a3b248"
server_key_version: "encryptionkey@0"
I20260929 11:03:01.725121 12145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "15e386b2b0c244168635658ca6299b3e"
format_stamp: "Formatted at 2026-09-29 11:03:01 on dist-test-slave-2lzq"
server_key: "ebf12624c1345453f5aa5e1bb7c773e1"
server_key_iv: "b280426589aaa4cce5d87f8310a3b248"
server_key_version: "encryptionkey@0"
I20260929 11:03:01.726611 12145 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260929 11:03:01.727459 12159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:01.727713 12145 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:03:01.727835 12145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "15e386b2b0c244168635658ca6299b3e"
format_stamp: "Formatted at 2026-09-29 11:03:01 on dist-test-slave-2lzq"
server_key: "ebf12624c1345453f5aa5e1bb7c773e1"
server_key_iv: "b280426589aaa4cce5d87f8310a3b248"
server_key_version: "encryptionkey@0"
I20260929 11:03:01.727911 12145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:01.745388 12145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:01.745729 12145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:01.745883 12145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:01.749642 12145 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:35657
I20260929 11:03:01.749730 12211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:35657 every 8 connection(s)
I20260929 11:03:01.750121 12145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:03:01.751113 12212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:01.753393 12212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15e386b2b0c244168635658ca6299b3e: Bootstrap starting.
I20260929 11:03:01.753976 12212 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15e386b2b0c244168635658ca6299b3e: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:01.754295 12212 log.cc:824] T 00000000000000000000000000000000 P 15e386b2b0c244168635658ca6299b3e: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:01.754961 12212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15e386b2b0c244168635658ca6299b3e: No bootstrap required, opened a new log
I20260929 11:03:01.755848 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 12145
I20260929 11:03:01.755955 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 11:03:01.756152 27107 external_mini_cluster.cc:1428] Setting key c1db0c0eeb1e7e79df8074319ded59cb
I20260929 11:03:01.757081 12212 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15e386b2b0c244168635658ca6299b3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15e386b2b0c244168635658ca6299b3e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 35657 } }
I20260929 11:03:01.757273 12212 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15e386b2b0c244168635658ca6299b3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:01.757315 12212 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15e386b2b0c244168635658ca6299b3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15e386b2b0c244168635658ca6299b3e, State: Initialized, Role: FOLLOWER
I20260929 11:03:01.757452 12212 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15e386b2b0c244168635658ca6299b3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15e386b2b0c244168635658ca6299b3e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 35657 } }
I20260929 11:03:01.757536 12212 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15e386b2b0c244168635658ca6299b3e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 11:03:01.757570 12212 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15e386b2b0c244168635658ca6299b3e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 11:03:01.757617 12212 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15e386b2b0c244168635658ca6299b3e [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:01.758373 12212 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15e386b2b0c244168635658ca6299b3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15e386b2b0c244168635658ca6299b3e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 35657 } }
I20260929 11:03:01.758481 12212 leader_election.cc:302] T 00000000000000000000000000000000 P 15e386b2b0c244168635658ca6299b3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15e386b2b0c244168635658ca6299b3e; no voters:
I20260929 11:03:01.758625 12212 leader_election.cc:288] T 00000000000000000000000000000000 P 15e386b2b0c244168635658ca6299b3e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 11:03:01.758711 12217 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15e386b2b0c244168635658ca6299b3e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:03:01.758860 12212 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15e386b2b0c244168635658ca6299b3e [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:03:01.758898 12217 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15e386b2b0c244168635658ca6299b3e [term 1 LEADER]: Becoming Leader. State: Replica: 15e386b2b0c244168635658ca6299b3e, State: Running, Role: LEADER
I20260929 11:03:01.759109 12217 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15e386b2b0c244168635658ca6299b3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15e386b2b0c244168635658ca6299b3e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 35657 } }
I20260929 11:03:01.759495 12218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15e386b2b0c244168635658ca6299b3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15e386b2b0c244168635658ca6299b3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15e386b2b0c244168635658ca6299b3e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 35657 } } }
I20260929 11:03:01.759572 12218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15e386b2b0c244168635658ca6299b3e [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:01.759722 12219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15e386b2b0c244168635658ca6299b3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15e386b2b0c244168635658ca6299b3e. Latest consensus state: current_term: 1 leader_uuid: "15e386b2b0c244168635658ca6299b3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15e386b2b0c244168635658ca6299b3e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 35657 } } }
I20260929 11:03:01.759778 12219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15e386b2b0c244168635658ca6299b3e [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:01.760187 12226 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:03:01.760766 12226 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:03:01.762310 12226 catalog_manager.cc:1347] Generated new cluster ID: fe27533df59d4d598237ffdb0d611e3e
I20260929 11:03:01.762372 12226 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 11:03:01.762809 12216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:01.756713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51900 (local address 127.26.120.254:35657)
0929 11:03:01.756920 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:01.756927 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:01.756951 (+ 24us) server_negotiation.cc:407] Connection header received
0929 11:03:01.757044 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:01.757049 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:01.757101 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:01.757221 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:01.757501 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.759326 (+ 1825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.760528 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.761126 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.761259 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:01.761987 (+ 728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:01.761990 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:01.761991 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:01.762216 (+ 225us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:01.762258 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:01.762274 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:01.762343 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:01.762424 (+ 81us) server_negotiation.cc:299] Negotiation successful
0929 11:03:01.762606 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":88,"threads_started":1}
I20260929 11:03:01.776537 12226 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 11:03:01.777104 12226 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:03:01.783751 12226 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15e386b2b0c244168635658ca6299b3e: Generated new TSK 0
I20260929 11:03:01.783926 12226 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 11:03:01.785772 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.193:0
--local_ip_for_outbound_sockets=127.26.120.193
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:35657
--builtin_ntp_servers=127.26.120.212:44131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 11:03:01.867223 12236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:01.867442 12236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:01.867463 12236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:01.868628 12236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:01.868669 12236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:01.868695 12236 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.193
I20260929 11:03:01.870045 12236 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:44131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:35657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.193
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:01.870339 12236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:01.870642 12236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:01.876776 12241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:01.876786 12242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:01.876860 12244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:01.877136 12236 server_base.cc:1034] running on GCE node
I20260929 11:03:01.877367 12236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:01.877610 12236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:01.878793 12236 hybrid_clock.cc:648] HybridClock initialized: now 1790679781878771 us; error 36 us; skew 500 ppm
I20260929 11:03:01.880141 12236 webserver.cc:460] Webserver started at http://127.26.120.193:41631/ using document root <none> and password file <none>
I20260929 11:03:01.880402 12236 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:01.880470 12236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:01.880614 12236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:01.881700 12236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-0/data/instance:
uuid: "096bcced3b5c4739a11af544cdd491ba"
format_stamp: "Formatted at 2026-09-29 11:03:01 on dist-test-slave-2lzq"
server_key: "4f0bcf3fb53d8a613a8819425999d8c2"
server_key_iv: "8b831cf15c7a3b3638c61961fe52313b"
server_key_version: "encryptionkey@0"
I20260929 11:03:01.882129 12236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance:
uuid: "096bcced3b5c4739a11af544cdd491ba"
format_stamp: "Formatted at 2026-09-29 11:03:01 on dist-test-slave-2lzq"
server_key: "4f0bcf3fb53d8a613a8819425999d8c2"
server_key_iv: "8b831cf15c7a3b3638c61961fe52313b"
server_key_version: "encryptionkey@0"
I20260929 11:03:01.883466 12236 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 11:03:01.884254 12250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:01.884493 12236 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 11:03:01.884603 12236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-0/wal
uuid: "096bcced3b5c4739a11af544cdd491ba"
format_stamp: "Formatted at 2026-09-29 11:03:01 on dist-test-slave-2lzq"
server_key: "4f0bcf3fb53d8a613a8819425999d8c2"
server_key_iv: "8b831cf15c7a3b3638c61961fe52313b"
server_key_version: "encryptionkey@0"
I20260929 11:03:01.884696 12236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:01.901991 12236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:01.902359 12236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:01.902520 12236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:01.902768 12236 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:03:01.903103 12236 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:03:01.903156 12236 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:01.903198 12236 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:03:01.903229 12236 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:01.909112 12236 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.193:45311
I20260929 11:03:01.909207 12363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.193:45311 every 8 connection(s)
I20260929 11:03:01.909550 12236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
I20260929 11:03:01.910795 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 12236
I20260929 11:03:01.910931 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance
I20260929 11:03:01.911118 27107 external_mini_cluster.cc:1428] Setting key 6521e5159f17a04b10a2336873b3f2e8
I20260929 11:03:01.911998 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.194:0
--local_ip_for_outbound_sockets=127.26.120.194
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:35657
--builtin_ntp_servers=127.26.120.212:44131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 11:03:01.915851 12216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:01.910468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:48099 (local address 127.26.120.254:35657)
0929 11:03:01.910556 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:01.910559 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:01.910803 (+ 244us) server_negotiation.cc:407] Connection header received
0929 11:03:01.911427 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:01.911428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:01.911444 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:01.911496 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:01.912233 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.912738 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.914226 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.914446 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.914507 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:01.915503 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:01.915505 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:01.915505 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:01.915511 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:01.915532 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:01.915537 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:01.915601 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:01.915728 (+ 127us) server_negotiation.cc:299] Negotiation successful
0929 11:03:01.915761 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 11:03:01.916118 12366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:01.910444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:35657 (local address 127.26.120.193:48099)
0929 11:03:01.910641 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:01.910683 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:01.911316 (+ 633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:01.911567 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:01.911571 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:01.911580 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:03:01.912142 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:01.912151 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.912835 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.912837 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:01.914138 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:01.914142 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:01.914543 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:01.914544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:01.914704 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:01.915304 (+ 600us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 11:03:01.915305 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:03:01.915345 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 11:03:01.915375 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:03:01.915617 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:03:01.915624 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:03:01.915628 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 11:03:01.915708 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 11:03:01.915841 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":96,"threads_started":1}
I20260929 11:03:01.916695 12364 heartbeater.cc:347] Connected to a master server at 127.26.120.254:35657
I20260929 11:03:01.916814 12364 heartbeater.cc:464] Registering TS with master...
I20260929 11:03:01.917032 12364 heartbeater.cc:511] Master 127.26.120.254:35657 requested a full tablet report, sending...
I20260929 11:03:01.917579 12176 ts_manager.cc:194] Registered new tserver with Master: 096bcced3b5c4739a11af544cdd491ba (127.26.120.193:45311)
I20260929 11:03:01.918282 12176 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.120.193:48099
W20260929 11:03:01.995488 12367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:01.995692 12367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:01.995714 12367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:01.996901 12367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:01.996943 12367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:01.996969 12367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.194
I20260929 11:03:01.998368 12367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:44131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:35657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.194
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:01.998572 12367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:01.998828 12367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 11:03:02.003818 12367 server_base.cc:1034] running on GCE node
W20260929 11:03:02.003774 12372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:02.003790 12373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:02.003780 12375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:02.004248 12367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:02.004508 12367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:02.005693 12367 hybrid_clock.cc:648] HybridClock initialized: now 1790679782005671 us; error 42 us; skew 500 ppm
I20260929 11:03:02.007128 12367 webserver.cc:460] Webserver started at http://127.26.120.194:34083/ using document root <none> and password file <none>
I20260929 11:03:02.007386 12367 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:02.007462 12367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:02.007612 12367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:02.008677 12367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-1/data/instance:
uuid: "dcef53da1e6544b293b0d3f6ccc3938c"
format_stamp: "Formatted at 2026-09-29 11:03:02 on dist-test-slave-2lzq"
server_key: "9dbb6550e1160559731049f1d3d97f37"
server_key_iv: "175ececb650355b664505e912d16bb5d"
server_key_version: "encryptionkey@0"
I20260929 11:03:02.009049 12367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance:
uuid: "dcef53da1e6544b293b0d3f6ccc3938c"
format_stamp: "Formatted at 2026-09-29 11:03:02 on dist-test-slave-2lzq"
server_key: "9dbb6550e1160559731049f1d3d97f37"
server_key_iv: "175ececb650355b664505e912d16bb5d"
server_key_version: "encryptionkey@0"
I20260929 11:03:02.010376 12367 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 11:03:02.011056 12381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:02.011222 12367 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:02.011324 12367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-1/wal
uuid: "dcef53da1e6544b293b0d3f6ccc3938c"
format_stamp: "Formatted at 2026-09-29 11:03:02 on dist-test-slave-2lzq"
server_key: "9dbb6550e1160559731049f1d3d97f37"
server_key_iv: "175ececb650355b664505e912d16bb5d"
server_key_version: "encryptionkey@0"
I20260929 11:03:02.011402 12367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:02.021070 12367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:02.021345 12367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:02.021493 12367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:02.021701 12367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:03:02.022007 12367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:03:02.022061 12367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:02.022176 12367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:03:02.022203 12367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:02.027971 12367 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.194:42465
I20260929 11:03:02.028044 12494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.194:42465 every 8 connection(s)
I20260929 11:03:02.028367 12367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
I20260929 11:03:02.033819 12216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:02.029107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:50765 (local address 127.26.120.254:35657)
0929 11:03:02.029152 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:02.029154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:02.029383 (+ 229us) server_negotiation.cc:407] Connection header received
0929 11:03:02.029816 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:02.029817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:02.029827 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:02.029893 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:02.030620 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:02.031130 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:02.032588 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:02.032781 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:02.032840 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:02.033591 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:02.033592 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:02.033593 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:02.033597 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:02.033611 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:02.033615 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:02.033658 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:02.033734 (+ 76us) server_negotiation.cc:299] Negotiation successful
0929 11:03:02.033759 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 11:03:02.034003 12497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:02.029055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:35657 (local address 127.26.120.194:50765)
0929 11:03:02.029231 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:02.029270 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:02.029732 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:02.029952 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:02.029955 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:02.029964 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:03:02.030540 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:02.030545 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:02.031224 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:02.031225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:02.032509 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:02.032512 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:02.032851 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:02.032852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:02.032964 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:02.033465 (+ 501us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 11:03:02.033465 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:03:02.033493 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 11:03:02.033514 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:03:02.033667 (+ 153us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:03:02.033670 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:03:02.033673 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 11:03:02.033724 (+ 51us) client_negotiation.cc:240] Negotiation successful
0929 11:03:02.033802 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":86,"threads_started":1}
I20260929 11:03:02.034540 12495 heartbeater.cc:347] Connected to a master server at 127.26.120.254:35657
I20260929 11:03:02.034631 12495 heartbeater.cc:464] Registering TS with master...
I20260929 11:03:02.034811 12495 heartbeater.cc:511] Master 127.26.120.254:35657 requested a full tablet report, sending...
I20260929 11:03:02.035171 12176 ts_manager.cc:194] Registered new tserver with Master: dcef53da1e6544b293b0d3f6ccc3938c (127.26.120.194:42465)
I20260929 11:03:02.035511 12176 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.120.194:50765
I20260929 11:03:02.037308 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 12367
I20260929 11:03:02.037439 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance
I20260929 11:03:02.037631 27107 external_mini_cluster.cc:1428] Setting key b7914f7acb3c2f73593a63dbf9f3551d
I20260929 11:03:02.038528 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.195:0
--local_ip_for_outbound_sockets=127.26.120.195
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:35657
--builtin_ntp_servers=127.26.120.212:44131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 11:03:02.118136 12498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:02.118358 12498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:02.118392 12498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:02.119546 12498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:02.119606 12498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:02.119648 12498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.195
I20260929 11:03:02.120967 12498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:44131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:35657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.195
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:02.121222 12498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:02.121501 12498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:02.126390 12503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:02.126520 12504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:02.126428 12506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:02.126644 12498 server_base.cc:1034] running on GCE node
I20260929 11:03:02.126873 12498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:02.127120 12498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:02.128324 12498 hybrid_clock.cc:648] HybridClock initialized: now 1790679782128316 us; error 37 us; skew 500 ppm
I20260929 11:03:02.129623 12498 webserver.cc:460] Webserver started at http://127.26.120.195:46399/ using document root <none> and password file <none>
I20260929 11:03:02.129887 12498 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:02.129957 12498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:02.130112 12498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:02.131278 12498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-2/data/instance:
uuid: "cb3c0397a4954df1ad360d2df30a42ab"
format_stamp: "Formatted at 2026-09-29 11:03:02 on dist-test-slave-2lzq"
server_key: "4c54ab837710d6cde251ca7f18c9507e"
server_key_iv: "542efd61eaace76f0267b8bbdabaa0c4"
server_key_version: "encryptionkey@0"
I20260929 11:03:02.131636 12498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance:
uuid: "cb3c0397a4954df1ad360d2df30a42ab"
format_stamp: "Formatted at 2026-09-29 11:03:02 on dist-test-slave-2lzq"
server_key: "4c54ab837710d6cde251ca7f18c9507e"
server_key_iv: "542efd61eaace76f0267b8bbdabaa0c4"
server_key_version: "encryptionkey@0"
I20260929 11:03:02.132872 12498 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 11:03:02.133584 12512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:02.133756 12498 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 11:03:02.133849 12498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-2/wal
uuid: "cb3c0397a4954df1ad360d2df30a42ab"
format_stamp: "Formatted at 2026-09-29 11:03:02 on dist-test-slave-2lzq"
server_key: "4c54ab837710d6cde251ca7f18c9507e"
server_key_iv: "542efd61eaace76f0267b8bbdabaa0c4"
server_key_version: "encryptionkey@0"
I20260929 11:03:02.133934 12498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:02.168365 12498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:02.168718 12498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:02.168890 12498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:02.169143 12498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:03:02.169502 12498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:03:02.169557 12498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:02.169598 12498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:03:02.169644 12498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:02.175536 12498 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.195:39823
I20260929 11:03:02.175580 12625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.195:39823 every 8 connection(s)
I20260929 11:03:02.175967 12498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
I20260929 11:03:02.181182 12216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:02.176824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:50267 (local address 127.26.120.254:35657)
0929 11:03:02.176893 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:02.176895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:02.177205 (+ 310us) server_negotiation.cc:407] Connection header received
0929 11:03:02.177597 (+ 392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:02.177598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:02.177609 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:02.177675 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:02.178457 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:02.178961 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:02.179905 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:02.180086 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:02.180138 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:02.180974 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:02.180974 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:02.180975 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:02.180979 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:02.180989 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:02.180993 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:02.181031 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:02.181103 (+ 72us) server_negotiation.cc:299] Negotiation successful
0929 11:03:02.181124 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 11:03:02.181382 12628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:02.176795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:35657 (local address 127.26.120.195:50267)
0929 11:03:02.176977 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:02.177019 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:02.177513 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:02.177746 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:02.177750 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:02.177758 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:03:02.178344 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:02.178349 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:02.179044 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:02.179045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:02.179826 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:02.179828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:02.180160 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:02.180161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:02.180272 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:02.180841 (+ 569us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 11:03:02.180841 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:03:02.180881 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 11:03:02.180904 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:03:02.181043 (+ 139us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:03:02.181045 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:03:02.181047 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 11:03:02.181095 (+ 48us) client_negotiation.cc:240] Negotiation successful
0929 11:03:02.181172 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":86,"threads_started":1}
I20260929 11:03:02.181850 12626 heartbeater.cc:347] Connected to a master server at 127.26.120.254:35657
I20260929 11:03:02.181950 12626 heartbeater.cc:464] Registering TS with master...
I20260929 11:03:02.182160 12626 heartbeater.cc:511] Master 127.26.120.254:35657 requested a full tablet report, sending...
I20260929 11:03:02.182516 12176 ts_manager.cc:194] Registered new tserver with Master: cb3c0397a4954df1ad360d2df30a42ab (127.26.120.195:39823)
I20260929 11:03:02.182883 12176 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.120.195:50267
I20260929 11:03:02.183660 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 12498
I20260929 11:03:02.183741 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance
I20260929 11:03:02.183954 27107 external_mini_cluster.cc:1428] Setting key 667e81a95d3afce7c87be05532e37a54
I20260929 11:03:02.184816 27107 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 11:03:02.189029 12216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:02.186170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51916 (local address 127.26.120.254:35657)
0929 11:03:02.186235 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:02.186237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:02.186388 (+ 151us) server_negotiation.cc:407] Connection header received
0929 11:03:02.186463 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:02.186464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:02.186474 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:02.186518 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:02.186683 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:02.187256 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:02.187997 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:02.188252 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:02.188306 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:02.188821 (+ 515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:02.188822 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:02.188822 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:02.188826 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:02.188838 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:02.188841 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:02.188889 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:02.188925 (+ 36us) server_negotiation.cc:299] Negotiation successful
0929 11:03:02.188945 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 11:03:02.190336 12176 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51916:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 11:03:02.191802 12176 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 11:03:02.197901 12639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:02.194248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48502 (local address 127.26.120.195:39823)
0929 11:03:02.194407 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:02.194412 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:02.194421 (+ 9us) server_negotiation.cc:407] Connection header received
0929 11:03:02.194452 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:02.194454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:02.194469 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:02.194554 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:03:02.194659 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:02.195423 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:02.197002 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:02.197508 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:02.197602 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:02.197698 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:02.197780 (+ 82us) server_negotiation.cc:299] Negotiation successful
0929 11:03:02.197815 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":79,"threads_started":1}
I20260929 11:03:02.197906 12635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:02.193788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:45311 (local address 127.0.0.1:42666)
0929 11:03:02.193937 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:02.193976 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:02.194039 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:02.194465 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:02.194466 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:02.194468 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:03:02.194513 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:02.194516 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:02.195886 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:02.195887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:02.196699 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:02.196701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:02.197644 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:02.197645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:02.197712 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:02.197718 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 11:03:02.197750 (+ 32us) client_negotiation.cc:240] Negotiation successful
0929 11:03:02.197810 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":51,"threads_started":1}
I20260929 11:03:02.198012 12634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:02.193788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:42465 (local address 127.0.0.1:57704)
0929 11:03:02.193954 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:02.193977 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:02.194037 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:02.194415 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:02.194418 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:02.194421 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:03:02.194484 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:02.194485 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:02.195851 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:02.195852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:02.196914 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:02.196916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:02.197760 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:02.197761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:02.197810 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:02.197813 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 11:03:02.197842 (+ 29us) client_negotiation.cc:240] Negotiation successful
0929 11:03:02.197890 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":27,"spinlock_wait_cycles":384,"thread_start_us":89,"threads_started":1}
I20260929 11:03:02.197908 12636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:02.193817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:39823 (local address 127.0.0.1:48502)
0929 11:03:02.194161 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:02.194171 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:02.194220 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:02.194559 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:02.194559 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:02.194561 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:03:02.194598 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:02.194599 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:02.195514 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:02.195515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:02.196733 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:02.196736 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:02.197600 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:02.197601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:02.197678 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:02.197692 (+ 14us) client_negotiation.cc:769] Sending connection context
0929 11:03:02.197768 (+ 76us) client_negotiation.cc:240] Negotiation successful
0929 11:03:02.197812 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":99,"threads_started":1}
I20260929 11:03:02.198217 12637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:02.194031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57704 (local address 127.26.120.194:42465)
0929 11:03:02.194216 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:02.194220 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:02.194231 (+ 11us) server_negotiation.cc:407] Connection header received
0929 11:03:02.194271 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:02.194274 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:02.194291 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:02.194403 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:03:02.194687 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:02.195759 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:02.196975 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:02.197673 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:02.197949 (+ 276us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:02.198050 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:02.198102 (+ 52us) server_negotiation.cc:299] Negotiation successful
0929 11:03:02.198139 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":117,"threads_started":1}
I20260929 11:03:02.198380 12638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:02.194043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42666 (local address 127.26.120.193:45311)
0929 11:03:02.194277 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:02.194280 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:02.194291 (+ 11us) server_negotiation.cc:407] Connection header received
0929 11:03:02.194328 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:02.194330 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:02.194344 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:02.194592 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:03:02.194653 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:02.195783 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:02.196786 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:02.197537 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:02.198129 (+ 592us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:02.198217 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:02.198241 (+ 24us) server_negotiation.cc:299] Negotiation successful
0929 11:03:02.198285 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":108,"threads_started":1}
I20260929 11:03:02.198773 12429 tablet_service.cc:1467] Processing CreateTablet for tablet 3d010c425dea46c191bde9cddd0fe890 (DEFAULT_TABLE table=test-table [id=ae0f79c0a1b04471b3a44731baf08866]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:03:02.199146 12429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d010c425dea46c191bde9cddd0fe890. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:02.199149 12298 tablet_service.cc:1467] Processing CreateTablet for tablet 3d010c425dea46c191bde9cddd0fe890 (DEFAULT_TABLE table=test-table [id=ae0f79c0a1b04471b3a44731baf08866]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:03:02.199205 12560 tablet_service.cc:1467] Processing CreateTablet for tablet 3d010c425dea46c191bde9cddd0fe890 (DEFAULT_TABLE table=test-table [id=ae0f79c0a1b04471b3a44731baf08866]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:03:02.199503 12560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d010c425dea46c191bde9cddd0fe890. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:02.199504 12298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d010c425dea46c191bde9cddd0fe890. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:02.201745 12640 tablet_bootstrap.cc:492] T 3d010c425dea46c191bde9cddd0fe890 P dcef53da1e6544b293b0d3f6ccc3938c: Bootstrap starting.
I20260929 11:03:02.202646 12640 tablet_bootstrap.cc:654] T 3d010c425dea46c191bde9cddd0fe890 P dcef53da1e6544b293b0d3f6ccc3938c: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:02.202798 12641 tablet_bootstrap.cc:492] T 3d010c425dea46c191bde9cddd0fe890 P cb3c0397a4954df1ad360d2df30a42ab: Bootstrap starting.
I20260929 11:03:02.203011 12640 log.cc:824] T 3d010c425dea46c191bde9cddd0fe890 P dcef53da1e6544b293b0d3f6ccc3938c: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:02.203462 12641 tablet_bootstrap.cc:654] T 3d010c425dea46c191bde9cddd0fe890 P cb3c0397a4954df1ad360d2df30a42ab: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:02.203784 12641 log.cc:824] T 3d010c425dea46c191bde9cddd0fe890 P cb3c0397a4954df1ad360d2df30a42ab: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:02.203814 12640 tablet_bootstrap.cc:492] T 3d010c425dea46c191bde9cddd0fe890 P dcef53da1e6544b293b0d3f6ccc3938c: No bootstrap required, opened a new log
I20260929 11:03:02.203878 12640 ts_tablet_manager.cc:1397] T 3d010c425dea46c191bde9cddd0fe890 P dcef53da1e6544b293b0d3f6ccc3938c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 11:03:02.204131 12642 tablet_bootstrap.cc:492] T 3d010c425dea46c191bde9cddd0fe890 P 096bcced3b5c4739a11af544cdd491ba: Bootstrap starting.
I20260929 11:03:02.204830 12642 tablet_bootstrap.cc:654] T 3d010c425dea46c191bde9cddd0fe890 P 096bcced3b5c4739a11af544cdd491ba: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:02.205152 12642 log.cc:824] T 3d010c425dea46c191bde9cddd0fe890 P 096bcced3b5c4739a11af544cdd491ba: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:02.205681 12641 tablet_bootstrap.cc:492] T 3d010c425dea46c191bde9cddd0fe890 P cb3c0397a4954df1ad360d2df30a42ab: No bootstrap required, opened a new log
I20260929 11:03:02.205749 12641 ts_tablet_manager.cc:1397] T 3d010c425dea46c191bde9cddd0fe890 P cb3c0397a4954df1ad360d2df30a42ab: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260929 11:03:02.206557 12642 tablet_bootstrap.cc:492] T 3d010c425dea46c191bde9cddd0fe890 P 096bcced3b5c4739a11af544cdd491ba: No bootstrap required, opened a new log
I20260929 11:03:02.206638 12642 ts_tablet_manager.cc:1397] T 3d010c425dea46c191bde9cddd0fe890 P 096bcced3b5c4739a11af544cdd491ba: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260929 11:03:02.206719 12640 raft_consensus.cc:348] T 3d010c425dea46c191bde9cddd0fe890 P dcef53da1e6544b293b0d3f6ccc3938c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "096bcced3b5c4739a11af544cdd491ba" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 45311 } } peers { permanent_uuid: "dcef53da1e6544b293b0d3f6ccc3938c" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 42465 } } peers { permanent_uuid: "cb3c0397a4954df1ad360d2df30a42ab" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 39823 } }
I20260929 11:03:02.206890 12640 raft_consensus.cc:374] T 3d010c425dea46c191bde9cddd0fe890 P dcef53da1e6544b293b0d3f6ccc3938c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:02.206961 12640 raft_consensus.cc:729] T 3d010c425dea46c191bde9cddd0fe890 P dcef53da1e6544b293b0d3f6ccc3938c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcef53da1e6544b293b0d3f6ccc3938c, State: Initialized, Role: FOLLOWER
I20260929 11:03:02.207093 12640 consensus_queue.cc:260] T 3d010c425dea46c191bde9cddd0fe890 P dcef53da1e6544b293b0d3f6ccc3938c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "096bcced3b5c4739a11af544cdd491ba" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 45311 } } peers { permanent_uuid: "dcef53da1e6544b293b0d3f6ccc3938c" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 42465 } } peers { permanent_uuid: "cb3c0397a4954df1ad360d2df30a42ab" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 39823 } }
I20260929 11:03:02.207350 12640 ts_tablet_manager.cc:1428] T 3d010c425dea46c191bde9cddd0fe890 P dcef53da1e6544b293b0d3f6ccc3938c: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260929 11:03:02.207500 12495 heartbeater.cc:503] Master 127.26.120.254:35657 was elected leader, sending a full tablet report...
I20260929 11:03:02.208169 12641 raft_consensus.cc:348] T 3d010c425dea46c191bde9cddd0fe890 P cb3c0397a4954df1ad360d2df30a42ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "096bcced3b5c4739a11af544cdd491ba" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 45311 } } peers { permanent_uuid: "dcef53da1e6544b293b0d3f6ccc3938c" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 42465 } } peers { permanent_uuid: "cb3c0397a4954df1ad360d2df30a42ab" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 39823 } }
I20260929 11:03:02.208364 12641 raft_consensus.cc:374] T 3d010c425dea46c191bde9cddd0fe890 P cb3c0397a4954df1ad360d2df30a42ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:02.208431 12641 raft_consensus.cc:729] T 3d010c425dea46c191bde9cddd0fe890 P cb3c0397a4954df1ad360d2df30a42ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb3c0397a4954df1ad360d2df30a42ab, State: Initialized, Role: FOLLOWER
I20260929 11:03:02.208572 12641 consensus_queue.cc:260] T 3d010c425dea46c191bde9cddd0fe890 P cb3c0397a4954df1ad360d2df30a42ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "096bcced3b5c4739a11af544cdd491ba" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 45311 } } peers { permanent_uuid: "dcef53da1e6544b293b0d3f6ccc3938c" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 42465 } } peers { permanent_uuid: "cb3c0397a4954df1ad360d2df30a42ab" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 39823 } }
I20260929 11:03:02.208845 12641 ts_tablet_manager.cc:1428] T 3d010c425dea46c191bde9cddd0fe890 P cb3c0397a4954df1ad360d2df30a42ab: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 11:03:02.208904 12626 heartbeater.cc:503] Master 127.26.120.254:35657 was elected leader, sending a full tablet report...
I20260929 11:03:02.209223 12642 raft_consensus.cc:348] T 3d010c425dea46c191bde9cddd0fe890 P 096bcced3b5c4739a11af544cdd491ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "096bcced3b5c4739a11af544cdd491ba" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 45311 } } peers { permanent_uuid: "dcef53da1e6544b293b0d3f6ccc3938c" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 42465 } } peers { permanent_uuid: "cb3c0397a4954df1ad360d2df30a42ab" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 39823 } }
I20260929 11:03:02.209394 12642 raft_consensus.cc:374] T 3d010c425dea46c191bde9cddd0fe890 P 096bcced3b5c4739a11af544cdd491ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:02.209432 12642 raft_consensus.cc:729] T 3d010c425dea46c191bde9cddd0fe890 P 096bcced3b5c4739a11af544cdd491ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 096bcced3b5c4739a11af544cdd491ba, State: Initialized, Role: FOLLOWER
I20260929 11:03:02.209544 12642 consensus_queue.cc:260] T 3d010c425dea46c191bde9cddd0fe890 P 096bcced3b5c4739a11af544cdd491ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "096bcced3b5c4739a11af544cdd491ba" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 45311 } } peers { permanent_uuid: "dcef53da1e6544b293b0d3f6ccc3938c" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 42465 } } peers { permanent_uuid: "cb3c0397a4954df1ad360d2df30a42ab" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 39823 } }
I20260929 11:03:02.209847 12642 ts_tablet_manager.cc:1428] T 3d010c425dea46c191bde9cddd0fe890 P 096bcced3b5c4739a11af544cdd491ba: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 11:03:02.209887 12364 heartbeater.cc:503] Master 127.26.120.254:35657 was elected leader, sending a full tablet report...
W20260929 11:03:02.278914 12496 tablet.cc:2369] T 3d010c425dea46c191bde9cddd0fe890 P dcef53da1e6544b293b0d3f6ccc3938c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 11:03:02.410259 12365 tablet.cc:2369] T 3d010c425dea46c191bde9cddd0fe890 P 096bcced3b5c4739a11af544cdd491ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 11:03:02.426590 12627 tablet.cc:2369] T 3d010c425dea46c191bde9cddd0fe890 P cb3c0397a4954df1ad360d2df30a42ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 11:03:02.523746 12647 raft_consensus.cc:482] T 3d010c425dea46c191bde9cddd0fe890 P cb3c0397a4954df1ad360d2df30a42ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 11:03:02.523872 12647 raft_consensus.cc:504] T 3d010c425dea46c191bde9cddd0fe890 P cb3c0397a4954df1ad360d2df30a42ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "096bcced3b5c4739a11af544cdd491ba" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 45311 } } peers { permanent_uuid: "dcef53da1e6544b293b0d3f6ccc3938c" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 42465 } } peers { permanent_uuid: "cb3c0397a4954df1ad360d2df30a42ab" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 39823 } }
I20260929 11:03:02.524289 12647 leader_election.cc:288] T 3d010c425dea46c191bde9cddd0fe890 P cb3c0397a4954df1ad360d2df30a42ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 096bcced3b5c4739a11af544cdd491ba (127.26.120.193:45311), dcef53da1e6544b293b0d3f6ccc3938c (127.26.120.194:42465)
I20260929 11:03:02.528265 12637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:02.524537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:40521 (local address 127.26.120.194:42465)
0929 11:03:02.524585 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:02.524587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:02.524597 (+ 10us) server_negotiation.cc:407] Connection header received
0929 11:03:02.524633 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:02.524634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:02.524644 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:02.524728 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:03:02.524929 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:02.525691 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:02.527287 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:02.527858 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:02.527940 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:02.527949 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:02.528156 (+ 207us) server_negotiation.cc:299] Negotiation successful
0929 11:03:02.528194 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 11:03:02.528311 12638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:02.524583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:41271 (local address 127.26.120.193:45311)
0929 11:03:02.524628 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:02.524630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:02.524688 (+ 58us) server_negotiation.cc:407] Connection header received
0929 11:03:02.524736 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:02.524736 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:02.524749 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:02.524849 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:03:02.525003 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:02.525701 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:02.527287 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:02.527858 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:02.527940 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:02.527949 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:02.528156 (+ 207us) server_negotiation.cc:299] Negotiation successful
0929 11:03:02.528195 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 11:03:02.528280 12628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:02.524438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:42465 (local address 127.26.120.195:40521)
0929 11:03:02.524502 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:02.524513 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:02.524557 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:02.524744 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:02.524745 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:02.524747 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:03:02.524820 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:02.524839 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:02.525794 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:02.525794 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:02.527175 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:02.527179 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:02.527969 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:02.527970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:02.528064 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:02.528071 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 11:03:02.528141 (+ 70us) client_negotiation.cc:240] Negotiation successful
0929 11:03:02.528195 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":104}
I20260929 11:03:02.528599 12649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:02.524439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:45311 (local address 127.26.120.195:41271)
0929 11:03:02.524617 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:02.524626 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:02.524677 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:02.524864 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:02.524865 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:02.524866 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:03:02.524926 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:02.524928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:02.525789 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:02.525790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:02.527176 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:02.527179 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:02.527969 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:02.527970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:02.528064 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:02.528072 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 11:03:02.528141 (+ 69us) client_negotiation.cc:240] Negotiation successful
0929 11:03:02.528194 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":50,"thread_start_us":103,"threads_started":1}
I20260929 11:03:02.528718 12449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d010c425dea46c191bde9cddd0fe890" candidate_uuid: "cb3c0397a4954df1ad360d2df30a42ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcef53da1e6544b293b0d3f6ccc3938c" is_pre_election: true
I20260929 11:03:02.528900 12449 raft_consensus.cc:2454] T 3d010c425dea46c191bde9cddd0fe890 P dcef53da1e6544b293b0d3f6ccc3938c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb3c0397a4954df1ad360d2df30a42ab in term 0.
I20260929 11:03:02.528951 12318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d010c425dea46c191bde9cddd0fe890" candidate_uuid: "cb3c0397a4954df1ad360d2df30a42ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "096bcced3b5c4739a11af544cdd491ba" is_pre_election: true
I20260929 11:03:02.529098 12318 raft_consensus.cc:2454] T 3d010c425dea46c191bde9cddd0fe890 P 096bcced3b5c4739a11af544cdd491ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb3c0397a4954df1ad360d2df30a42ab in term 0.
I20260929 11:03:02.529130 12513 leader_election.cc:302] T 3d010c425dea46c191bde9cddd0fe890 P cb3c0397a4954df1ad360d2df30a42ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb3c0397a4954df1ad360d2df30a42ab, dcef53da1e6544b293b0d3f6ccc3938c; no voters:
I20260929 11:03:02.529390 12647 raft_consensus.cc:2781] T 3d010c425dea46c191bde9cddd0fe890 P cb3c0397a4954df1ad360d2df30a42ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 11:03:02.529440 12647 raft_consensus.cc:482] T 3d010c425dea46c191bde9cddd0fe890 P cb3c0397a4954df1ad360d2df30a42ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 11:03:02.529457 12647 raft_consensus.cc:3037] T 3d010c425dea46c191bde9cddd0fe890 P cb3c0397a4954df1ad360d2df30a42ab [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:02.530182 12647 raft_consensus.cc:504] T 3d010c425dea46c191bde9cddd0fe890 P cb3c0397a4954df1ad360d2df30a42ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "096bcced3b5c4739a11af544cdd491ba" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 45311 } } peers { permanent_uuid: "dcef53da1e6544b293b0d3f6ccc3938c" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 42465 } } peers { permanent_uuid: "cb3c0397a4954df1ad360d2df30a42ab" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 39823 } }
I20260929 11:03:02.530329 12647 leader_election.cc:288] T 3d010c425dea46c191bde9cddd0fe890 P cb3c0397a4954df1ad360d2df30a42ab [CANDIDATE]: Term 1 election: Requested vote from peers 096bcced3b5c4739a11af544cdd491ba (127.26.120.193:45311), dcef53da1e6544b293b0d3f6ccc3938c (127.26.120.194:42465)
I20260929 11:03:02.530478 12318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d010c425dea46c191bde9cddd0fe890" candidate_uuid: "cb3c0397a4954df1ad360d2df30a42ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "096bcced3b5c4739a11af544cdd491ba"
I20260929 11:03:02.530511 12449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d010c425dea46c191bde9cddd0fe890" candidate_uuid: "cb3c0397a4954df1ad360d2df30a42ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcef53da1e6544b293b0d3f6ccc3938c"
I20260929 11:03:02.530576 12318 raft_consensus.cc:3037] T 3d010c425dea46c191bde9cddd0fe890 P 096bcced3b5c4739a11af544cdd491ba [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:02.530629 12449 raft_consensus.cc:3037] T 3d010c425dea46c191bde9cddd0fe890 P dcef53da1e6544b293b0d3f6ccc3938c [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:02.531324 12318 raft_consensus.cc:2454] T 3d010c425dea46c191bde9cddd0fe890 P 096bcced3b5c4739a11af544cdd491ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb3c0397a4954df1ad360d2df30a42ab in term 1.
I20260929 11:03:02.531337 12449 raft_consensus.cc:2454] T 3d010c425dea46c191bde9cddd0fe890 P dcef53da1e6544b293b0d3f6ccc3938c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb3c0397a4954df1ad360d2df30a42ab in term 1.
I20260929 11:03:02.531494 12513 leader_election.cc:302] T 3d010c425dea46c191bde9cddd0fe890 P cb3c0397a4954df1ad360d2df30a42ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb3c0397a4954df1ad360d2df30a42ab, dcef53da1e6544b293b0d3f6ccc3938c; no voters:
I20260929 11:03:02.531591 12647 raft_consensus.cc:2781] T 3d010c425dea46c191bde9cddd0fe890 P cb3c0397a4954df1ad360d2df30a42ab [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:03:02.531733 12647 raft_consensus.cc:686] T 3d010c425dea46c191bde9cddd0fe890 P cb3c0397a4954df1ad360d2df30a42ab [term 1 LEADER]: Becoming Leader. State: Replica: cb3c0397a4954df1ad360d2df30a42ab, State: Running, Role: LEADER
I20260929 11:03:02.531832 12647 consensus_queue.cc:237] T 3d010c425dea46c191bde9cddd0fe890 P cb3c0397a4954df1ad360d2df30a42ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "096bcced3b5c4739a11af544cdd491ba" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 45311 } } peers { permanent_uuid: "dcef53da1e6544b293b0d3f6ccc3938c" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 42465 } } peers { permanent_uuid: "cb3c0397a4954df1ad360d2df30a42ab" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 39823 } }
I20260929 11:03:02.532593 12176 catalog_manager.cc:5495] T 3d010c425dea46c191bde9cddd0fe890 P cb3c0397a4954df1ad360d2df30a42ab reported cstate change: term changed from 0 to 1, leader changed from <none> to cb3c0397a4954df1ad360d2df30a42ab (127.26.120.195). New cstate: current_term: 1 leader_uuid: "cb3c0397a4954df1ad360d2df30a42ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "096bcced3b5c4739a11af544cdd491ba" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 45311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dcef53da1e6544b293b0d3f6ccc3938c" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 42465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb3c0397a4954df1ad360d2df30a42ab" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 39823 } health_report { overall_health: HEALTHY } } }
I20260929 11:03:02.549809 12639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:02.547452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48516 (local address 127.26.120.195:39823)
0929 11:03:02.547494 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:02.547496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:02.547511 (+ 15us) server_negotiation.cc:407] Connection header received
0929 11:03:02.547549 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:02.547550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:02.547569 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:02.547648 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 11:03:02.547819 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:02.548302 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:02.549032 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:02.549257 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:02.549316 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:02.549478 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 11:03:02.549577 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 11:03:02.549635 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:02.549735 (+ 100us) server_negotiation.cc:299] Negotiation successful
0929 11:03:02.549750 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 11:03:02.551651 12318 raft_consensus.cc:1261] T 3d010c425dea46c191bde9cddd0fe890 P 096bcced3b5c4739a11af544cdd491ba [term 1 FOLLOWER]: Refusing update from remote peer cb3c0397a4954df1ad360d2df30a42ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 11:03:02.551678 12449 raft_consensus.cc:1261] T 3d010c425dea46c191bde9cddd0fe890 P dcef53da1e6544b293b0d3f6ccc3938c [term 1 FOLLOWER]: Refusing update from remote peer cb3c0397a4954df1ad360d2df30a42ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 11:03:02.552052 12647 consensus_queue.cc:1035] T 3d010c425dea46c191bde9cddd0fe890 P cb3c0397a4954df1ad360d2df30a42ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "096bcced3b5c4739a11af544cdd491ba" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 45311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 11:03:02.552212 12650 consensus_queue.cc:1035] T 3d010c425dea46c191bde9cddd0fe890 P cb3c0397a4954df1ad360d2df30a42ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "dcef53da1e6544b293b0d3f6ccc3938c" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 11:03:02.554549 12653 mvcc.cc:205] Tried to move back new op lower bound from 7334624389328916480 to 7334624389250879488. Current Snapshot: MvccSnapshot[applied={T|T < 7334624389328916480}]
I20260929 11:03:02.554805 12654 mvcc.cc:205] Tried to move back new op lower bound from 7334624389328916480 to 7334624389250879488. Current Snapshot: MvccSnapshot[applied={T|T < 7334624389328916480}]
I20260929 11:03:02.555377 12655 mvcc.cc:205] Tried to move back new op lower bound from 7334624389328916480 to 7334624389250879488. Current Snapshot: MvccSnapshot[applied={T|T < 7334624389328916480}]
I20260929 11:03:02.556749 12176 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51916:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 11:03:02.556851 12176 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51916:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 11:03:02.557682 12176 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 15e386b2b0c244168635658ca6299b3e: Sending DeleteTablet for 3 replicas of tablet 3d010c425dea46c191bde9cddd0fe890
I20260929 11:03:02.558002 12298 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d010c425dea46c191bde9cddd0fe890 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:03:02 UTC) from {username='slave'} at 127.0.0.1:42666
I20260929 11:03:02.558018 12429 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d010c425dea46c191bde9cddd0fe890 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:03:02 UTC) from {username='slave'} at 127.0.0.1:57704
I20260929 11:03:02.558322 12662 tablet_replica.cc:330] T 3d010c425dea46c191bde9cddd0fe890 P 096bcced3b5c4739a11af544cdd491ba: stopping tablet replica
I20260929 11:03:02.558322 12663 tablet_replica.cc:330] T 3d010c425dea46c191bde9cddd0fe890 P dcef53da1e6544b293b0d3f6ccc3938c: stopping tablet replica
I20260929 11:03:02.558314 12560 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d010c425dea46c191bde9cddd0fe890 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:03:02 UTC) from {username='slave'} at 127.0.0.1:48502
I20260929 11:03:02.558416 12663 raft_consensus.cc:2229] T 3d010c425dea46c191bde9cddd0fe890 P dcef53da1e6544b293b0d3f6ccc3938c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 11:03:02.558431 12662 raft_consensus.cc:2229] T 3d010c425dea46c191bde9cddd0fe890 P 096bcced3b5c4739a11af544cdd491ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 11:03:02.558512 12663 raft_consensus.cc:2258] T 3d010c425dea46c191bde9cddd0fe890 P dcef53da1e6544b293b0d3f6ccc3938c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 11:03:02.558549 12662 raft_consensus.cc:2258] T 3d010c425dea46c191bde9cddd0fe890 P 096bcced3b5c4739a11af544cdd491ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 11:03:02.558935 12664 tablet_replica.cc:330] T 3d010c425dea46c191bde9cddd0fe890 P cb3c0397a4954df1ad360d2df30a42ab: stopping tablet replica
I20260929 11:03:02.558962 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 12236
I20260929 11:03:02.559024 12662 ts_tablet_manager.cc:1905] T 3d010c425dea46c191bde9cddd0fe890 P 096bcced3b5c4739a11af544cdd491ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 11:03:02.559469 12664 raft_consensus.cc:2229] T 3d010c425dea46c191bde9cddd0fe890 P cb3c0397a4954df1ad360d2df30a42ab [term 1 LEADER]: Raft consensus shutting down.
I20260929 11:03:02.559588 12664 raft_consensus.cc:2258] T 3d010c425dea46c191bde9cddd0fe890 P cb3c0397a4954df1ad360d2df30a42ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 11:03:02.559943 12664 ts_tablet_manager.cc:1905] T 3d010c425dea46c191bde9cddd0fe890 P cb3c0397a4954df1ad360d2df30a42ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 11:03:02.561550 12664 ts_tablet_manager.cc:1918] T 3d010c425dea46c191bde9cddd0fe890 P cb3c0397a4954df1ad360d2df30a42ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 11:03:02.561604 12664 log.cc:1196] T 3d010c425dea46c191bde9cddd0fe890 P cb3c0397a4954df1ad360d2df30a42ab: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-2/wal/wals/3d010c425dea46c191bde9cddd0fe890
I20260929 11:03:02.561846 12664 ts_tablet_manager.cc:1939] T 3d010c425dea46c191bde9cddd0fe890 P cb3c0397a4954df1ad360d2df30a42ab: Deleting consensus metadata
I20260929 11:03:02.562202 12162 catalog_manager.cc:4841] TS cb3c0397a4954df1ad360d2df30a42ab (127.26.120.195:39823): tablet 3d010c425dea46c191bde9cddd0fe890 (table test-table [id=ae0f79c0a1b04471b3a44731baf08866]) successfully deleted
I20260929 11:03:02.566437 12663 ts_tablet_manager.cc:1905] T 3d010c425dea46c191bde9cddd0fe890 P dcef53da1e6544b293b0d3f6ccc3938c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260929 11:03:02.567322 12163 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.120.193:45311 (error 108)
W20260929 11:03:02.567878 12636 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 11:03:02.567707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:45311 (local address 127.0.0.1:42676)
0929 11:03:02.567782 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:02.567843 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.193:45311: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":46}
I20260929 11:03:02.568001 12663 ts_tablet_manager.cc:1918] T 3d010c425dea46c191bde9cddd0fe890 P dcef53da1e6544b293b0d3f6ccc3938c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260929 11:03:02.568044 12163 catalog_manager.cc:4584] TS 096bcced3b5c4739a11af544cdd491ba (127.26.120.193:45311): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3d010c425dea46c191bde9cddd0fe890: Network error: Client connection negotiation failed: client connection to 127.26.120.193:45311: connect: Connection reset by peer (error 104)
I20260929 11:03:02.568085 12663 log.cc:1196] T 3d010c425dea46c191bde9cddd0fe890 P dcef53da1e6544b293b0d3f6ccc3938c: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790679549986039-27107-0/minicluster-data/ts-1/wal/wals/3d010c425dea46c191bde9cddd0fe890
I20260929 11:03:02.568287 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 12367
I20260929 11:03:02.572933 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 12498
I20260929 11:03:02.573045 12636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:02.572919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:42465 (local address 127.0.0.1:57716)
0929 11:03:02.572963 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:02.572994 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.194:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260929 11:03:02.577679 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 12145
2026-09-29T11:03:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (973 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 11:03:02 dist-test-slave-2lzq krb5kdc[12668](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 11:03:02 dist-test-slave-2lzq krb5kdc[12668](info): set up 2 sockets
Sep 29 11:03:02 dist-test-slave-2lzq krb5kdc[12668](info): commencing operation
krb5kdc: starting...
W20260929 11:03:03.610152 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 11:03:03 dist-test-slave-2lzq krb5kdc[12668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T11:03:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T11:03:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.120.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
I20260929 11:03:03.726871 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:38655
--webserver_interface=127.26.120.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:39851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:38655
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:03:03.806015 12681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:03.806283 12681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:03.806322 12681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:03.807489 12681 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:03:03.807559 12681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:03.807582 12681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:03.807605 12681 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:03:03.807633 12681 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:03:03.809082 12681 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:39851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:38655
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:38655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:03.809365 12681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:03.809592 12681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:03.814396 12687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:03.814407 12689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:03.814391 12686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:03.814634 12681 server_base.cc:1034] running on GCE node
I20260929 11:03:03.814880 12681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:03.815205 12681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:03.816377 12681 hybrid_clock.cc:648] HybridClock initialized: now 1790679783816361 us; error 44 us; skew 500 ppm
Sep 29 11:03:03 dist-test-slave-2lzq krb5kdc[12668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679783, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:03:03.823551 12681 init.cc:377] Logged in from keytab as kudu/127.26.120.254@KRBTEST.COM (short username kudu)
I20260929 11:03:03.824004 12681 webserver.cc:460] Webserver started at http://127.26.120.254:37021/ using document root <none> and password file <none>
I20260929 11:03:03.824235 12681 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:03.824302 12681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:03.824463 12681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:03.825546 12681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "74ee9cec78714b25adc4cd99aaf5956e"
format_stamp: "Formatted at 2026-09-29 11:03:03 on dist-test-slave-2lzq"
server_key: "dfab4810c8b4211675027d52ab5c5cd8"
server_key_iv: "c5d34c402795481d07bc0abc1b7a8e3e"
server_key_version: "encryptionkey@0"
I20260929 11:03:03.825918 12681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "74ee9cec78714b25adc4cd99aaf5956e"
format_stamp: "Formatted at 2026-09-29 11:03:03 on dist-test-slave-2lzq"
server_key: "dfab4810c8b4211675027d52ab5c5cd8"
server_key_iv: "c5d34c402795481d07bc0abc1b7a8e3e"
server_key_version: "encryptionkey@0"
I20260929 11:03:03.827298 12681 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 11:03:03.828048 12696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:03.828217 12681 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:03.828316 12681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "74ee9cec78714b25adc4cd99aaf5956e"
format_stamp: "Formatted at 2026-09-29 11:03:03 on dist-test-slave-2lzq"
server_key: "dfab4810c8b4211675027d52ab5c5cd8"
server_key_iv: "c5d34c402795481d07bc0abc1b7a8e3e"
server_key_version: "encryptionkey@0"
I20260929 11:03:03.828418 12681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:03.846664 12681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:03.849062 12681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:03.849234 12681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:03.853205 12681 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:38655
I20260929 11:03:03.853259 12748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:38655 every 8 connection(s)
I20260929 11:03:03.853644 12681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:03:03.854586 12749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:03.856828 12749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74ee9cec78714b25adc4cd99aaf5956e: Bootstrap starting.
I20260929 11:03:03.857383 12749 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74ee9cec78714b25adc4cd99aaf5956e: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:03.857622 12749 log.cc:824] T 00000000000000000000000000000000 P 74ee9cec78714b25adc4cd99aaf5956e: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:03.858393 12749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74ee9cec78714b25adc4cd99aaf5956e: No bootstrap required, opened a new log
I20260929 11:03:03.859920 12749 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74ee9cec78714b25adc4cd99aaf5956e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74ee9cec78714b25adc4cd99aaf5956e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 38655 } }
I20260929 11:03:03.860056 12749 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74ee9cec78714b25adc4cd99aaf5956e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:03.860078 12749 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74ee9cec78714b25adc4cd99aaf5956e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74ee9cec78714b25adc4cd99aaf5956e, State: Initialized, Role: FOLLOWER
I20260929 11:03:03.860212 12749 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74ee9cec78714b25adc4cd99aaf5956e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74ee9cec78714b25adc4cd99aaf5956e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 38655 } }
I20260929 11:03:03.860296 12749 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74ee9cec78714b25adc4cd99aaf5956e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 11:03:03.860317 12749 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74ee9cec78714b25adc4cd99aaf5956e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 11:03:03.860347 12749 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74ee9cec78714b25adc4cd99aaf5956e [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:03.860836 12749 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74ee9cec78714b25adc4cd99aaf5956e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74ee9cec78714b25adc4cd99aaf5956e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 38655 } }
I20260929 11:03:03.860924 12749 leader_election.cc:302] T 00000000000000000000000000000000 P 74ee9cec78714b25adc4cd99aaf5956e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74ee9cec78714b25adc4cd99aaf5956e; no voters:
I20260929 11:03:03.861078 12749 leader_election.cc:288] T 00000000000000000000000000000000 P 74ee9cec78714b25adc4cd99aaf5956e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 11:03:03.861114 12752 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74ee9cec78714b25adc4cd99aaf5956e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:03:03.861238 12752 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74ee9cec78714b25adc4cd99aaf5956e [term 1 LEADER]: Becoming Leader. State: Replica: 74ee9cec78714b25adc4cd99aaf5956e, State: Running, Role: LEADER
I20260929 11:03:03.861334 12749 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74ee9cec78714b25adc4cd99aaf5956e [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:03:03.861403 12752 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74ee9cec78714b25adc4cd99aaf5956e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74ee9cec78714b25adc4cd99aaf5956e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 38655 } }
I20260929 11:03:03.861750 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 12681
I20260929 11:03:03.861796 12754 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74ee9cec78714b25adc4cd99aaf5956e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74ee9cec78714b25adc4cd99aaf5956e. Latest consensus state: current_term: 1 leader_uuid: "74ee9cec78714b25adc4cd99aaf5956e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74ee9cec78714b25adc4cd99aaf5956e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 38655 } } }
I20260929 11:03:03.861871 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 11:03:03.861882 12754 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74ee9cec78714b25adc4cd99aaf5956e [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:03.861950 12753 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74ee9cec78714b25adc4cd99aaf5956e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74ee9cec78714b25adc4cd99aaf5956e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74ee9cec78714b25adc4cd99aaf5956e" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 38655 } } }
I20260929 11:03:03.862036 12753 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74ee9cec78714b25adc4cd99aaf5956e [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:03.862109 27107 external_mini_cluster.cc:1428] Setting key f581623ae29e0b3c5f285778817676f2
W20260929 11:03:03.863204 12769 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 74ee9cec78714b25adc4cd99aaf5956e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 11:03:03.863262 12769 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 11:03:03.863306 12760 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:03:03.863459 12760 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:03:03.864827 12760 catalog_manager.cc:1347] Generated new cluster ID: f83659f48fd142e6a26f2b9b402dcb0e
I20260929 11:03:03.864918 12760 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 11:03:03 dist-test-slave-2lzq krb5kdc[12668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:03:03.875171 12760 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 11:03:03.875664 12770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:03.862738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45616 (local address 127.26.120.254:38655)
0929 11:03:03.863024 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:03.863034 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:03.863068 (+ 34us) server_negotiation.cc:407] Connection header received
0929 11:03:03.863187 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:03.863194 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:03.863263 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:03.863437 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:03.864195 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:03.866954 (+ 2759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:03.868051 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:03.868657 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:03.868749 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:03.871719 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:03.871724 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:03.871724 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:03.871738 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:03.873764 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:03.874213 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:03.874216 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:03.874216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:03.874332 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:03.874546 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:03.874547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:03.874548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:03.874842 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:03.875017 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:03.875122 (+ 105us) server_negotiation.cc:299] Negotiation successful
0929 11:03:03.875416 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":97,"threads_started":1}
I20260929 11:03:03.875679 12760 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:03:03.890648 12760 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74ee9cec78714b25adc4cd99aaf5956e: Generated new TSK 0
I20260929 11:03:03.890867 12760 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.120.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab.
Entry for principal kudu/127.26.120.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab.
I20260929 11:03:03.923460 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.193:0
--local_ip_for_outbound_sockets=127.26.120.193
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:38655
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:39851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:03:04.019765 12775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:04.019968 12775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:04.019992 12775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:04.021066 12775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:04.021102 12775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:04.021128 12775 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.193
I20260929 11:03:04.022624 12775 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:39851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:38655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.193
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:04.022816 12775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:04.023007 12775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:04.027963 12781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:04.028002 12780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:04.027969 12783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:04.028048 12775 server_base.cc:1034] running on GCE node
I20260929 11:03:04.028411 12775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:04.028672 12775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:04.029846 12775 hybrid_clock.cc:648] HybridClock initialized: now 1790679784029825 us; error 40 us; skew 500 ppm
Sep 29 11:03:04 dist-test-slave-2lzq krb5kdc[12668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679784, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:03:04.032552 12775 init.cc:377] Logged in from keytab as kudu/127.26.120.193@KRBTEST.COM (short username kudu)
I20260929 11:03:04.032982 12775 webserver.cc:460] Webserver started at http://127.26.120.193:37571/ using document root <none> and password file <none>
I20260929 11:03:04.033206 12775 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:04.033274 12775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:04.033427 12775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:04.034541 12775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-0/data/instance:
uuid: "7e54a9510ff14895a4d054d30070b6d1"
format_stamp: "Formatted at 2026-09-29 11:03:04 on dist-test-slave-2lzq"
server_key: "82eb32455768a29331469b532444f2f1"
server_key_iv: "40f039be2b81c0d938ed1c62c965ce6b"
server_key_version: "encryptionkey@0"
I20260929 11:03:04.034895 12775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance:
uuid: "7e54a9510ff14895a4d054d30070b6d1"
format_stamp: "Formatted at 2026-09-29 11:03:04 on dist-test-slave-2lzq"
server_key: "82eb32455768a29331469b532444f2f1"
server_key_iv: "40f039be2b81c0d938ed1c62c965ce6b"
server_key_version: "encryptionkey@0"
I20260929 11:03:04.036166 12775 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260929 11:03:04.036876 12790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:04.037042 12775 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 11:03:04.037139 12775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-0/wal
uuid: "7e54a9510ff14895a4d054d30070b6d1"
format_stamp: "Formatted at 2026-09-29 11:03:04 on dist-test-slave-2lzq"
server_key: "82eb32455768a29331469b532444f2f1"
server_key_iv: "40f039be2b81c0d938ed1c62c965ce6b"
server_key_version: "encryptionkey@0"
I20260929 11:03:04.037227 12775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:04.073290 12775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:04.075327 12775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:04.075515 12775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:04.075760 12775 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:03:04.076086 12775 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:03:04.076141 12775 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:04.076175 12775 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:03:04.076216 12775 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:04.082159 12775 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.193:44199
I20260929 11:03:04.082212 12903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.193:44199 every 8 connection(s)
I20260929 11:03:04.082623 12775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
Sep 29 11:03:04 dist-test-slave-2lzq krb5kdc[12668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679784, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:03:04.089648 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 12775
I20260929 11:03:04.089807 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance
I20260929 11:03:04.090045 27107 external_mini_cluster.cc:1428] Setting key a8c1186f7d4288b91b6cb1790e6ed8db
I20260929 11:03:04.092671 12770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:04.083332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:32889 (local address 127.26.120.254:38655)
0929 11:03:04.083418 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:04.083420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:04.083629 (+ 209us) server_negotiation.cc:407] Connection header received
0929 11:03:04.084110 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:04.084111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:04.084126 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:04.084181 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:04.085162 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.085675 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.086739 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.086940 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.086999 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:04.089232 (+ 2233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:04.089233 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:04.089234 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:04.089242 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:04.090942 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:04.091480 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:04.091480 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:04.091481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:04.091532 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:04.091821 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:04.091822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:04.091822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:04.091931 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:04.091980 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:04.092418 (+ 438us) server_negotiation.cc:299] Negotiation successful
0929 11:03:04.092537 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 11:03:04.092836 12906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:04.083316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:38655 (local address 127.26.120.193:32889)
0929 11:03:04.083503 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:04.083542 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:04.084022 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:04.084263 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:04.084266 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:04.084846 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:03:04.085066 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:04.085071 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.085757 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.085758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:04.086652 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:04.086655 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.087021 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.087022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:04.087129 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:04.087645 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:03:04.087653 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:03:04.089118 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:03:04.091021 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:04.091031 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:04.091034 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:04.091401 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:04.091601 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:04.091602 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:04.091602 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:04.091760 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:04.091995 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:03:04.092001 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:03:04.092213 (+ 212us) client_negotiation.cc:769] Sending connection context
0929 11:03:04.092387 (+ 174us) client_negotiation.cc:240] Negotiation successful
0929 11:03:04.092602 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":101,"threads_started":1}
I20260929 11:03:04.093395 12904 heartbeater.cc:347] Connected to a master server at 127.26.120.254:38655
I20260929 11:03:04.093511 12904 heartbeater.cc:464] Registering TS with master...
I20260929 11:03:04.093717 12904 heartbeater.cc:511] Master 127.26.120.254:38655 requested a full tablet report, sending...
I20260929 11:03:04.094305 12713 ts_manager.cc:194] Registered new tserver with Master: 7e54a9510ff14895a4d054d30070b6d1 (127.26.120.193:44199)
I20260929 11:03:04.094985 12713 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.193@KRBTEST.COM'} at 127.26.120.193:32889
WARNING: no policy specified for kudu/127.26.120.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab.
Entry for principal kudu/127.26.120.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab.
I20260929 11:03:04.116195 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.194:0
--local_ip_for_outbound_sockets=127.26.120.194
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:38655
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:39851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:03:04.197165 12909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:04.197427 12909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:04.197465 12909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:04.198639 12909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:04.198710 12909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:04.198755 12909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.194
I20260929 11:03:04.200235 12909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:39851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:38655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.194
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:04.200500 12909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:04.200732 12909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:04.205780 12914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:04.205845 12917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:04.205986 12915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:04.205817 12909 server_base.cc:1034] running on GCE node
I20260929 11:03:04.206218 12909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:04.206481 12909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:04.207653 12909 hybrid_clock.cc:648] HybridClock initialized: now 1790679784207641 us; error 42 us; skew 500 ppm
Sep 29 11:03:04 dist-test-slave-2lzq krb5kdc[12668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679784, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:03:04.210041 12909 init.cc:377] Logged in from keytab as kudu/127.26.120.194@KRBTEST.COM (short username kudu)
I20260929 11:03:04.210566 12909 webserver.cc:460] Webserver started at http://127.26.120.194:41285/ using document root <none> and password file <none>
I20260929 11:03:04.210820 12909 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:04.210894 12909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:04.211046 12909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:04.212060 12909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-1/data/instance:
uuid: "7242a9321b844797b0cd9f9e7a5921a7"
format_stamp: "Formatted at 2026-09-29 11:03:04 on dist-test-slave-2lzq"
server_key: "8cd648cc7d4d22d88f577e0570da059e"
server_key_iv: "d8c3ba8e0991235cf449c5e6592b5de4"
server_key_version: "encryptionkey@0"
I20260929 11:03:04.212407 12909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance:
uuid: "7242a9321b844797b0cd9f9e7a5921a7"
format_stamp: "Formatted at 2026-09-29 11:03:04 on dist-test-slave-2lzq"
server_key: "8cd648cc7d4d22d88f577e0570da059e"
server_key_iv: "d8c3ba8e0991235cf449c5e6592b5de4"
server_key_version: "encryptionkey@0"
I20260929 11:03:04.213691 12909 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260929 11:03:04.214481 12924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:04.214649 12909 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:04.214781 12909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-1/wal
uuid: "7242a9321b844797b0cd9f9e7a5921a7"
format_stamp: "Formatted at 2026-09-29 11:03:04 on dist-test-slave-2lzq"
server_key: "8cd648cc7d4d22d88f577e0570da059e"
server_key_iv: "d8c3ba8e0991235cf449c5e6592b5de4"
server_key_version: "encryptionkey@0"
I20260929 11:03:04.214869 12909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:04.236924 12909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:04.238994 12909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:04.239148 12909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:04.239390 12909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:03:04.239759 12909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:03:04.239820 12909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:04.239863 12909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:03:04.239899 12909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:04.245674 12909 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.194:38953
I20260929 11:03:04.245728 13037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.194:38953 every 8 connection(s)
I20260929 11:03:04.246030 12909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
I20260929 11:03:04.251258 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 12909
I20260929 11:03:04.251422 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance
I20260929 11:03:04.251637 27107 external_mini_cluster.cc:1428] Setting key a6fc62e6576708f2a57d542f5af02fb4
Sep 29 11:03:04 dist-test-slave-2lzq krb5kdc[12668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679784, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:03:04.255712 12770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:04.246783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:44693 (local address 127.26.120.254:38655)
0929 11:03:04.246825 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:04.246827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:04.247036 (+ 209us) server_negotiation.cc:407] Connection header received
0929 11:03:04.247574 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:04.247576 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:04.247589 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:04.247653 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:04.248577 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.249098 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.250144 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.250365 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.250451 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:04.252728 (+ 2277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:04.252729 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:04.252730 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:04.252738 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:04.254306 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:04.254708 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:04.254709 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:04.254709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:04.254772 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:04.255016 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:04.255017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:04.255017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:04.255153 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:04.255226 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:04.255536 (+ 310us) server_negotiation.cc:299] Negotiation successful
0929 11:03:04.255624 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 11:03:04.255904 13040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:04.246780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:38655 (local address 127.26.120.194:44693)
0929 11:03:04.246916 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:04.246953 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:04.247465 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:04.247726 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:04.247730 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:04.248285 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:03:04.248479 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:04.248483 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.249196 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.249197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:04.250007 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:04.250009 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.250470 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.250471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:04.250600 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:04.251059 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:03:04.251066 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:03:04.252628 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:03:04.254396 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:04.254402 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:04.254404 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:04.254633 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:04.254847 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:04.254847 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:04.254847 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:04.254948 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:04.255232 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:03:04.255239 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:03:04.255374 (+ 135us) client_negotiation.cc:769] Sending connection context
0929 11:03:04.255503 (+ 129us) client_negotiation.cc:240] Negotiation successful
0929 11:03:04.255657 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":73,"threads_started":1}
I20260929 11:03:04.256309 13038 heartbeater.cc:347] Connected to a master server at 127.26.120.254:38655
I20260929 11:03:04.256409 13038 heartbeater.cc:464] Registering TS with master...
I20260929 11:03:04.256608 13038 heartbeater.cc:511] Master 127.26.120.254:38655 requested a full tablet report, sending...
I20260929 11:03:04.256991 12713 ts_manager.cc:194] Registered new tserver with Master: 7242a9321b844797b0cd9f9e7a5921a7 (127.26.120.194:38953)
I20260929 11:03:04.257383 12713 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.194@KRBTEST.COM'} at 127.26.120.194:44693
WARNING: no policy specified for kudu/127.26.120.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab.
Entry for principal kudu/127.26.120.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab.
I20260929 11:03:04.276650 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.195:0
--local_ip_for_outbound_sockets=127.26.120.195
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:38655
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:39851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:03:04.356554 13043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:04.356769 13043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:04.356794 13043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:04.357895 13043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:04.357933 13043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:04.357959 13043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.195
I20260929 11:03:04.359467 13043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:39851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:38655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.195
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:04.359661 13043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:04.359838 13043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:04.364980 13049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:04.364964 13051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:04.365064 13043 server_base.cc:1034] running on GCE node
W20260929 11:03:04.364964 13048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:04.365338 13043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:04.365585 13043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:04.366720 13043 hybrid_clock.cc:648] HybridClock initialized: now 1790679784366707 us; error 48 us; skew 500 ppm
Sep 29 11:03:04 dist-test-slave-2lzq krb5kdc[12668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679784, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:03:04.369187 13043 init.cc:377] Logged in from keytab as kudu/127.26.120.195@KRBTEST.COM (short username kudu)
I20260929 11:03:04.369568 13043 webserver.cc:460] Webserver started at http://127.26.120.195:40263/ using document root <none> and password file <none>
I20260929 11:03:04.369750 13043 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:04.369788 13043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:04.369875 13043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:04.371017 13043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-2/data/instance:
uuid: "429a7629064f450b9d31429705424332"
format_stamp: "Formatted at 2026-09-29 11:03:04 on dist-test-slave-2lzq"
server_key: "a5577ba0d28fa8894662a1da0bd36fb1"
server_key_iv: "1436baac7058e82492e52ea35145d38f"
server_key_version: "encryptionkey@0"
I20260929 11:03:04.371313 13043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance:
uuid: "429a7629064f450b9d31429705424332"
format_stamp: "Formatted at 2026-09-29 11:03:04 on dist-test-slave-2lzq"
server_key: "a5577ba0d28fa8894662a1da0bd36fb1"
server_key_iv: "1436baac7058e82492e52ea35145d38f"
server_key_version: "encryptionkey@0"
I20260929 11:03:04.372550 13043 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 11:03:04.373230 13058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:04.373383 13043 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 11:03:04.373474 13043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-2/wal
uuid: "429a7629064f450b9d31429705424332"
format_stamp: "Formatted at 2026-09-29 11:03:04 on dist-test-slave-2lzq"
server_key: "a5577ba0d28fa8894662a1da0bd36fb1"
server_key_iv: "1436baac7058e82492e52ea35145d38f"
server_key_version: "encryptionkey@0"
I20260929 11:03:04.373528 13043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:04.386186 13043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:04.388044 13043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:04.388152 13043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:04.388341 13043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:03:04.388593 13043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:03:04.388623 13043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:04.388644 13043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:03:04.388657 13043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:04.394627 13043 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.195:37455
I20260929 11:03:04.394711 13171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.195:37455 every 8 connection(s)
I20260929 11:03:04.395021 13043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
Sep 29 11:03:04 dist-test-slave-2lzq krb5kdc[12668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679784, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:03:04.401371 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 13043
I20260929 11:03:04.401474 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance
I20260929 11:03:04.401674 27107 external_mini_cluster.cc:1428] Setting key 8f7d518af8a582a36c488bf021f9459b
I20260929 11:03:04.404757 12770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:04.395761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:35715 (local address 127.26.120.254:38655)
0929 11:03:04.395827 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:04.395829 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:04.396029 (+ 200us) server_negotiation.cc:407] Connection header received
0929 11:03:04.396521 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:04.396522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:04.396534 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:04.396593 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:04.397568 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.398182 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.399143 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.399353 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.399411 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:04.401711 (+ 2300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:04.401712 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:04.401713 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:04.401720 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:04.403370 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:04.403791 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:04.403792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:04.403792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:04.403851 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:04.404106 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:04.404107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:04.404108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:04.404269 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:04.404336 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:04.404615 (+ 279us) server_negotiation.cc:299] Negotiation successful
0929 11:03:04.404690 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 11:03:04.404884 13174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:04.395751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:38655 (local address 127.26.120.195:35715)
0929 11:03:04.395926 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:04.395961 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:04.396428 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:04.396664 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:04.396667 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:04.397246 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:03:04.397448 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:04.397453 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.398265 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.398265 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:04.399065 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:04.399067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.399430 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.399430 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:04.399530 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:04.399981 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:03:04.399989 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:03:04.401587 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:03:04.403447 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:04.403454 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:04.403455 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:04.403684 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:04.403922 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:04.403923 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:04.403923 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:04.404024 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:04.404339 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:03:04.404344 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:03:04.404471 (+ 127us) client_negotiation.cc:769] Sending connection context
0929 11:03:04.404558 (+ 87us) client_negotiation.cc:240] Negotiation successful
0929 11:03:04.404685 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":91,"threads_started":1}
I20260929 11:03:04.405377 13172 heartbeater.cc:347] Connected to a master server at 127.26.120.254:38655
I20260929 11:03:04.405478 13172 heartbeater.cc:464] Registering TS with master...
I20260929 11:03:04.405622 13172 heartbeater.cc:511] Master 127.26.120.254:38655 requested a full tablet report, sending...
I20260929 11:03:04.405973 12713 ts_manager.cc:194] Registered new tserver with Master: 429a7629064f450b9d31429705424332 (127.26.120.195:37455)
I20260929 11:03:04.406386 12713 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.195@KRBTEST.COM'} at 127.26.120.195:35715
I20260929 11:03:04.413870 27107 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 11:03:04.421582 12770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:04.415347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45618 (local address 127.26.120.254:38655)
0929 11:03:04.415392 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:04.415402 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:04.415500 (+ 98us) server_negotiation.cc:407] Connection header received
0929 11:03:04.415542 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:04.415543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:04.415552 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:04.415603 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:04.416178 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.416655 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.417517 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.417702 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.417754 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:04.419112 (+ 1358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:04.419114 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:04.419114 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:04.419120 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:04.420486 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:04.420847 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:04.420848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:04.420848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:04.420908 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:04.421154 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:04.421154 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:04.421154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:04.421278 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:04.421356 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:04.421472 (+ 116us) server_negotiation.cc:299] Negotiation successful
0929 11:03:04.421520 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 11:03:04.423259 12713 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45618:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 11:03:04.424857 12713 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 11:03:04.431615 13185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:04.427344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:44199 (local address 127.0.0.1:39790)
0929 11:03:04.427806 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:04.427814 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:04.427889 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:04.428178 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:04.428181 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:04.428184 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:03:04.428268 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:04.428270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.429226 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.429227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:04.430325 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:04.430328 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.431333 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.431334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:04.431390 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:04.431400 (+ 10us) client_negotiation.cc:769] Sending connection context
0929 11:03:04.431483 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 11:03:04.431540 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"mutex_wait_us":1,"thread_start_us":115,"threads_started":1}
I20260929 11:03:04.431627 13188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:04.427598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39790 (local address 127.26.120.193:44199)
0929 11:03:04.427751 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:04.427755 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:04.427878 (+ 123us) server_negotiation.cc:407] Connection header received
0929 11:03:04.427985 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:04.427987 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:04.428006 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:04.428085 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:03:04.428468 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.429126 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.430422 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.431163 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.431247 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:04.431305 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:04.431500 (+ 195us) server_negotiation.cc:299] Negotiation successful
0929 11:03:04.431540 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":80,"threads_started":1}
I20260929 11:03:04.432298 13183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:04.427343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:38953 (local address 127.0.0.1:48232)
0929 11:03:04.427989 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:04.427999 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:04.428041 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:04.428313 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:04.428314 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:04.428316 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:03:04.428360 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:04.428361 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.429766 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.429767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:04.430980 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:04.430983 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.432039 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.432041 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:04.432111 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:04.432116 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 11:03:04.432169 (+ 53us) client_negotiation.cc:240] Negotiation successful
0929 11:03:04.432213 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"mutex_wait_us":71,"thread_start_us":111,"threads_started":1}
I20260929 11:03:04.432507 13184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:04.427338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:37455 (local address 127.0.0.1:33864)
0929 11:03:04.427754 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:04.427798 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:04.427878 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:04.428183 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:04.428184 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:04.428186 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:03:04.428264 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:04.428265 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.429790 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.429791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:04.430903 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:04.430906 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.432263 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.432264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:04.432329 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:04.432333 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 11:03:04.432391 (+ 58us) client_negotiation.cc:240] Negotiation successful
0929 11:03:04.432439 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":5,"thread_start_us":76,"threads_started":1}
I20260929 11:03:04.432641 13186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:04.427524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48232 (local address 127.26.120.194:38953)
0929 11:03:04.427706 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:04.427711 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:04.428106 (+ 395us) server_negotiation.cc:407] Connection header received
0929 11:03:04.428150 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:04.428152 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:04.428169 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:04.428236 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:03:04.428445 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.429646 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.431053 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.431873 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.431947 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:04.432001 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:04.432481 (+ 480us) server_negotiation.cc:299] Negotiation successful
0929 11:03:04.432531 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":90,"threads_started":1}
I20260929 11:03:04.432650 12838 tablet_service.cc:1467] Processing CreateTablet for tablet 42e11d8cc46f41ac8b859ad7f259c559 (DEFAULT_TABLE table=test-table [id=8a812701897e4389ac34374af263310e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:03:04.432837 13187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:04.427536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33864 (local address 127.26.120.195:37455)
0929 11:03:04.427777 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:04.427783 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:04.427854 (+ 71us) server_negotiation.cc:407] Connection header received
0929 11:03:04.428005 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:04.428008 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:04.428026 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:04.428101 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:03:04.428354 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.429639 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.431000 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.432067 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.432137 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:04.432228 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:04.432686 (+ 458us) server_negotiation.cc:299] Negotiation successful
0929 11:03:04.432744 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":91,"threads_started":1}
I20260929 11:03:04.433027 12838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42e11d8cc46f41ac8b859ad7f259c559. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:04.433287 12972 tablet_service.cc:1467] Processing CreateTablet for tablet 42e11d8cc46f41ac8b859ad7f259c559 (DEFAULT_TABLE table=test-table [id=8a812701897e4389ac34374af263310e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:03:04.433466 13106 tablet_service.cc:1467] Processing CreateTablet for tablet 42e11d8cc46f41ac8b859ad7f259c559 (DEFAULT_TABLE table=test-table [id=8a812701897e4389ac34374af263310e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:03:04.433609 12972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42e11d8cc46f41ac8b859ad7f259c559. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:04.433794 13106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42e11d8cc46f41ac8b859ad7f259c559. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:04.436520 13189 tablet_bootstrap.cc:492] T 42e11d8cc46f41ac8b859ad7f259c559 P 429a7629064f450b9d31429705424332: Bootstrap starting.
I20260929 11:03:04.437122 13191 tablet_bootstrap.cc:492] T 42e11d8cc46f41ac8b859ad7f259c559 P 7242a9321b844797b0cd9f9e7a5921a7: Bootstrap starting.
I20260929 11:03:04.437353 13189 tablet_bootstrap.cc:654] T 42e11d8cc46f41ac8b859ad7f259c559 P 429a7629064f450b9d31429705424332: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:04.437891 13191 tablet_bootstrap.cc:654] T 42e11d8cc46f41ac8b859ad7f259c559 P 7242a9321b844797b0cd9f9e7a5921a7: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:04.437970 13189 log.cc:824] T 42e11d8cc46f41ac8b859ad7f259c559 P 429a7629064f450b9d31429705424332: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:04.438431 13190 tablet_bootstrap.cc:492] T 42e11d8cc46f41ac8b859ad7f259c559 P 7e54a9510ff14895a4d054d30070b6d1: Bootstrap starting.
I20260929 11:03:04.438587 13191 log.cc:824] T 42e11d8cc46f41ac8b859ad7f259c559 P 7242a9321b844797b0cd9f9e7a5921a7: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:04.438860 13189 tablet_bootstrap.cc:492] T 42e11d8cc46f41ac8b859ad7f259c559 P 429a7629064f450b9d31429705424332: No bootstrap required, opened a new log
I20260929 11:03:04.438948 13189 ts_tablet_manager.cc:1397] T 42e11d8cc46f41ac8b859ad7f259c559 P 429a7629064f450b9d31429705424332: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260929 11:03:04.439178 13190 tablet_bootstrap.cc:654] T 42e11d8cc46f41ac8b859ad7f259c559 P 7e54a9510ff14895a4d054d30070b6d1: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:04.439455 13191 tablet_bootstrap.cc:492] T 42e11d8cc46f41ac8b859ad7f259c559 P 7242a9321b844797b0cd9f9e7a5921a7: No bootstrap required, opened a new log
I20260929 11:03:04.439781 13190 log.cc:824] T 42e11d8cc46f41ac8b859ad7f259c559 P 7e54a9510ff14895a4d054d30070b6d1: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:04.439805 13191 ts_tablet_manager.cc:1397] T 42e11d8cc46f41ac8b859ad7f259c559 P 7242a9321b844797b0cd9f9e7a5921a7: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260929 11:03:04.440589 13190 tablet_bootstrap.cc:492] T 42e11d8cc46f41ac8b859ad7f259c559 P 7e54a9510ff14895a4d054d30070b6d1: No bootstrap required, opened a new log
I20260929 11:03:04.440665 13190 ts_tablet_manager.cc:1397] T 42e11d8cc46f41ac8b859ad7f259c559 P 7e54a9510ff14895a4d054d30070b6d1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 11:03:04.440943 13189 raft_consensus.cc:348] T 42e11d8cc46f41ac8b859ad7f259c559 P 429a7629064f450b9d31429705424332 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e54a9510ff14895a4d054d30070b6d1" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44199 } } peers { permanent_uuid: "7242a9321b844797b0cd9f9e7a5921a7" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 38953 } } peers { permanent_uuid: "429a7629064f450b9d31429705424332" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 37455 } }
I20260929 11:03:04.441134 13189 raft_consensus.cc:374] T 42e11d8cc46f41ac8b859ad7f259c559 P 429a7629064f450b9d31429705424332 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:04.441167 13189 raft_consensus.cc:729] T 42e11d8cc46f41ac8b859ad7f259c559 P 429a7629064f450b9d31429705424332 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 429a7629064f450b9d31429705424332, State: Initialized, Role: FOLLOWER
I20260929 11:03:04.441285 13189 consensus_queue.cc:260] T 42e11d8cc46f41ac8b859ad7f259c559 P 429a7629064f450b9d31429705424332 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e54a9510ff14895a4d054d30070b6d1" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44199 } } peers { permanent_uuid: "7242a9321b844797b0cd9f9e7a5921a7" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 38953 } } peers { permanent_uuid: "429a7629064f450b9d31429705424332" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 37455 } }
I20260929 11:03:04.441555 13189 ts_tablet_manager.cc:1428] T 42e11d8cc46f41ac8b859ad7f259c559 P 429a7629064f450b9d31429705424332: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260929 11:03:04.441596 13172 heartbeater.cc:503] Master 127.26.120.254:38655 was elected leader, sending a full tablet report...
I20260929 11:03:04.441813 13191 raft_consensus.cc:348] T 42e11d8cc46f41ac8b859ad7f259c559 P 7242a9321b844797b0cd9f9e7a5921a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e54a9510ff14895a4d054d30070b6d1" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44199 } } peers { permanent_uuid: "7242a9321b844797b0cd9f9e7a5921a7" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 38953 } } peers { permanent_uuid: "429a7629064f450b9d31429705424332" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 37455 } }
I20260929 11:03:04.442014 13191 raft_consensus.cc:374] T 42e11d8cc46f41ac8b859ad7f259c559 P 7242a9321b844797b0cd9f9e7a5921a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:04.442101 13191 raft_consensus.cc:729] T 42e11d8cc46f41ac8b859ad7f259c559 P 7242a9321b844797b0cd9f9e7a5921a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7242a9321b844797b0cd9f9e7a5921a7, State: Initialized, Role: FOLLOWER
I20260929 11:03:04.442278 13191 consensus_queue.cc:260] T 42e11d8cc46f41ac8b859ad7f259c559 P 7242a9321b844797b0cd9f9e7a5921a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e54a9510ff14895a4d054d30070b6d1" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44199 } } peers { permanent_uuid: "7242a9321b844797b0cd9f9e7a5921a7" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 38953 } } peers { permanent_uuid: "429a7629064f450b9d31429705424332" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 37455 } }
I20260929 11:03:04.442613 13038 heartbeater.cc:503] Master 127.26.120.254:38655 was elected leader, sending a full tablet report...
I20260929 11:03:04.442698 13190 raft_consensus.cc:348] T 42e11d8cc46f41ac8b859ad7f259c559 P 7e54a9510ff14895a4d054d30070b6d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e54a9510ff14895a4d054d30070b6d1" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44199 } } peers { permanent_uuid: "7242a9321b844797b0cd9f9e7a5921a7" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 38953 } } peers { permanent_uuid: "429a7629064f450b9d31429705424332" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 37455 } }
I20260929 11:03:04.442829 13191 ts_tablet_manager.cc:1428] T 42e11d8cc46f41ac8b859ad7f259c559 P 7242a9321b844797b0cd9f9e7a5921a7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 11:03:04.442819 13190 raft_consensus.cc:374] T 42e11d8cc46f41ac8b859ad7f259c559 P 7e54a9510ff14895a4d054d30070b6d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:04.442873 13190 raft_consensus.cc:729] T 42e11d8cc46f41ac8b859ad7f259c559 P 7e54a9510ff14895a4d054d30070b6d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e54a9510ff14895a4d054d30070b6d1, State: Initialized, Role: FOLLOWER
I20260929 11:03:04.442965 13190 consensus_queue.cc:260] T 42e11d8cc46f41ac8b859ad7f259c559 P 7e54a9510ff14895a4d054d30070b6d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e54a9510ff14895a4d054d30070b6d1" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44199 } } peers { permanent_uuid: "7242a9321b844797b0cd9f9e7a5921a7" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 38953 } } peers { permanent_uuid: "429a7629064f450b9d31429705424332" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 37455 } }
I20260929 11:03:04.443271 12904 heartbeater.cc:503] Master 127.26.120.254:38655 was elected leader, sending a full tablet report...
I20260929 11:03:04.443634 13190 ts_tablet_manager.cc:1428] T 42e11d8cc46f41ac8b859ad7f259c559 P 7e54a9510ff14895a4d054d30070b6d1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260929 11:03:04.496625 13039 tablet.cc:2369] T 42e11d8cc46f41ac8b859ad7f259c559 P 7242a9321b844797b0cd9f9e7a5921a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 11:03:04.583369 12905 tablet.cc:2369] T 42e11d8cc46f41ac8b859ad7f259c559 P 7e54a9510ff14895a4d054d30070b6d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 11:03:04.637039 13195 raft_consensus.cc:482] T 42e11d8cc46f41ac8b859ad7f259c559 P 429a7629064f450b9d31429705424332 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 11:03:04.637183 13195 raft_consensus.cc:504] T 42e11d8cc46f41ac8b859ad7f259c559 P 429a7629064f450b9d31429705424332 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e54a9510ff14895a4d054d30070b6d1" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44199 } } peers { permanent_uuid: "7242a9321b844797b0cd9f9e7a5921a7" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 38953 } } peers { permanent_uuid: "429a7629064f450b9d31429705424332" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 37455 } }
I20260929 11:03:04.637643 13195 leader_election.cc:288] T 42e11d8cc46f41ac8b859ad7f259c559 P 429a7629064f450b9d31429705424332 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e54a9510ff14895a4d054d30070b6d1 (127.26.120.193:44199), 7242a9321b844797b0cd9f9e7a5921a7 (127.26.120.194:38953)
I20260929 11:03:04.640672 13188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:04.637917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:35397 (local address 127.26.120.193:44199)
0929 11:03:04.637962 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:04.637964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:04.638001 (+ 37us) server_negotiation.cc:407] Connection header received
0929 11:03:04.638111 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:04.638112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:04.638121 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:04.638192 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:03:04.638372 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.638858 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.639835 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.640298 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.640382 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:04.640392 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:04.640570 (+ 178us) server_negotiation.cc:299] Negotiation successful
0929 11:03:04.640609 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 11:03:04.640672 13174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:04.637845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:44199 (local address 127.26.120.195:35397)
0929 11:03:04.637940 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:04.637952 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:04.638018 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:04.638216 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:04.638217 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:04.638219 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:03:04.638294 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:04.638296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.638940 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.638941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:04.639732 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:04.639734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.640391 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.640392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:04.640485 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:04.640487 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 11:03:04.640560 (+ 73us) client_negotiation.cc:240] Negotiation successful
0929 11:03:04.640599 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260929 11:03:04.640733 13186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:04.637943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:43743 (local address 127.26.120.194:38953)
0929 11:03:04.637995 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:04.637997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:04.638268 (+ 271us) server_negotiation.cc:407] Connection header received
0929 11:03:04.638330 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:04.638331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:04.638341 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:04.638423 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:03:04.638580 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.639042 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.639985 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.640375 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.640464 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:04.640473 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:04.640624 (+ 151us) server_negotiation.cc:299] Negotiation successful
0929 11:03:04.640662 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260929 11:03:04.640784 13198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:04.637851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:38953 (local address 127.26.120.195:43743)
0929 11:03:04.638203 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:04.638211 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:04.638274 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:04.638429 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:04.638430 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:04.638432 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:03:04.638504 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:04.638505 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.639125 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.639126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:04.639897 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:04.639899 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.640464 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.640465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:04.640536 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:04.640539 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 11:03:04.640615 (+ 76us) client_negotiation.cc:240] Negotiation successful
0929 11:03:04.640653 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":129,"thread_start_us":156,"threads_started":1}
I20260929 11:03:04.641124 12992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42e11d8cc46f41ac8b859ad7f259c559" candidate_uuid: "429a7629064f450b9d31429705424332" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7242a9321b844797b0cd9f9e7a5921a7" is_pre_election: true
I20260929 11:03:04.641122 12858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42e11d8cc46f41ac8b859ad7f259c559" candidate_uuid: "429a7629064f450b9d31429705424332" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e54a9510ff14895a4d054d30070b6d1" is_pre_election: true
I20260929 11:03:04.641281 12992 raft_consensus.cc:2454] T 42e11d8cc46f41ac8b859ad7f259c559 P 7242a9321b844797b0cd9f9e7a5921a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 429a7629064f450b9d31429705424332 in term 0.
I20260929 11:03:04.641284 12858 raft_consensus.cc:2454] T 42e11d8cc46f41ac8b859ad7f259c559 P 7e54a9510ff14895a4d054d30070b6d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 429a7629064f450b9d31429705424332 in term 0.
I20260929 11:03:04.641515 13060 leader_election.cc:302] T 42e11d8cc46f41ac8b859ad7f259c559 P 429a7629064f450b9d31429705424332 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 429a7629064f450b9d31429705424332, 7e54a9510ff14895a4d054d30070b6d1; no voters:
I20260929 11:03:04.641669 13195 raft_consensus.cc:2781] T 42e11d8cc46f41ac8b859ad7f259c559 P 429a7629064f450b9d31429705424332 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 11:03:04.641722 13195 raft_consensus.cc:482] T 42e11d8cc46f41ac8b859ad7f259c559 P 429a7629064f450b9d31429705424332 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 11:03:04.641747 13195 raft_consensus.cc:3037] T 42e11d8cc46f41ac8b859ad7f259c559 P 429a7629064f450b9d31429705424332 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:04.642413 13195 raft_consensus.cc:504] T 42e11d8cc46f41ac8b859ad7f259c559 P 429a7629064f450b9d31429705424332 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e54a9510ff14895a4d054d30070b6d1" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44199 } } peers { permanent_uuid: "7242a9321b844797b0cd9f9e7a5921a7" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 38953 } } peers { permanent_uuid: "429a7629064f450b9d31429705424332" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 37455 } }
I20260929 11:03:04.642549 13195 leader_election.cc:288] T 42e11d8cc46f41ac8b859ad7f259c559 P 429a7629064f450b9d31429705424332 [CANDIDATE]: Term 1 election: Requested vote from peers 7e54a9510ff14895a4d054d30070b6d1 (127.26.120.193:44199), 7242a9321b844797b0cd9f9e7a5921a7 (127.26.120.194:38953)
I20260929 11:03:04.642721 12992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42e11d8cc46f41ac8b859ad7f259c559" candidate_uuid: "429a7629064f450b9d31429705424332" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7242a9321b844797b0cd9f9e7a5921a7"
I20260929 11:03:04.642714 12858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42e11d8cc46f41ac8b859ad7f259c559" candidate_uuid: "429a7629064f450b9d31429705424332" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e54a9510ff14895a4d054d30070b6d1"
I20260929 11:03:04.642791 12992 raft_consensus.cc:3037] T 42e11d8cc46f41ac8b859ad7f259c559 P 7242a9321b844797b0cd9f9e7a5921a7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:04.642791 12858 raft_consensus.cc:3037] T 42e11d8cc46f41ac8b859ad7f259c559 P 7e54a9510ff14895a4d054d30070b6d1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:04.643651 12858 raft_consensus.cc:2454] T 42e11d8cc46f41ac8b859ad7f259c559 P 7e54a9510ff14895a4d054d30070b6d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 429a7629064f450b9d31429705424332 in term 1.
I20260929 11:03:04.643651 12992 raft_consensus.cc:2454] T 42e11d8cc46f41ac8b859ad7f259c559 P 7242a9321b844797b0cd9f9e7a5921a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 429a7629064f450b9d31429705424332 in term 1.
I20260929 11:03:04.643868 13059 leader_election.cc:302] T 42e11d8cc46f41ac8b859ad7f259c559 P 429a7629064f450b9d31429705424332 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 429a7629064f450b9d31429705424332, 7242a9321b844797b0cd9f9e7a5921a7; no voters:
I20260929 11:03:04.643980 13195 raft_consensus.cc:2781] T 42e11d8cc46f41ac8b859ad7f259c559 P 429a7629064f450b9d31429705424332 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:03:04.644148 13195 raft_consensus.cc:686] T 42e11d8cc46f41ac8b859ad7f259c559 P 429a7629064f450b9d31429705424332 [term 1 LEADER]: Becoming Leader. State: Replica: 429a7629064f450b9d31429705424332, State: Running, Role: LEADER
I20260929 11:03:04.644254 13195 consensus_queue.cc:237] T 42e11d8cc46f41ac8b859ad7f259c559 P 429a7629064f450b9d31429705424332 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e54a9510ff14895a4d054d30070b6d1" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44199 } } peers { permanent_uuid: "7242a9321b844797b0cd9f9e7a5921a7" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 38953 } } peers { permanent_uuid: "429a7629064f450b9d31429705424332" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 37455 } }
I20260929 11:03:04.645107 12712 catalog_manager.cc:5495] T 42e11d8cc46f41ac8b859ad7f259c559 P 429a7629064f450b9d31429705424332 reported cstate change: term changed from 0 to 1, leader changed from <none> to 429a7629064f450b9d31429705424332 (127.26.120.195). New cstate: current_term: 1 leader_uuid: "429a7629064f450b9d31429705424332" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e54a9510ff14895a4d054d30070b6d1" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7242a9321b844797b0cd9f9e7a5921a7" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 38953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "429a7629064f450b9d31429705424332" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 37455 } health_report { overall_health: HEALTHY } } }
W20260929 11:03:04.645607 13173 tablet.cc:2369] T 42e11d8cc46f41ac8b859ad7f259c559 P 429a7629064f450b9d31429705424332: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 11:03:04.663421 13187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:04.660877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33880 (local address 127.26.120.195:37455)
0929 11:03:04.660955 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:04.660957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:04.660966 (+ 9us) server_negotiation.cc:407] Connection header received
0929 11:03:04.661008 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:04.661008 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:04.661026 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:04.661086 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 11:03:04.661236 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.661781 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.662722 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.662906 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.662966 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:04.663097 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 11:03:04.663180 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 11:03:04.663287 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:04.663335 (+ 48us) server_negotiation.cc:299] Negotiation successful
0929 11:03:04.663345 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 11:03:04.665215 12992 raft_consensus.cc:1261] T 42e11d8cc46f41ac8b859ad7f259c559 P 7242a9321b844797b0cd9f9e7a5921a7 [term 1 FOLLOWER]: Refusing update from remote peer 429a7629064f450b9d31429705424332: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 11:03:04.665189 12858 raft_consensus.cc:1261] T 42e11d8cc46f41ac8b859ad7f259c559 P 7e54a9510ff14895a4d054d30070b6d1 [term 1 FOLLOWER]: Refusing update from remote peer 429a7629064f450b9d31429705424332: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 11:03:04.665580 13195 consensus_queue.cc:1035] T 42e11d8cc46f41ac8b859ad7f259c559 P 429a7629064f450b9d31429705424332 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7242a9321b844797b0cd9f9e7a5921a7" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 38953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 11:03:04.665736 13199 consensus_queue.cc:1035] T 42e11d8cc46f41ac8b859ad7f259c559 P 429a7629064f450b9d31429705424332 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e54a9510ff14895a4d054d30070b6d1" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 44199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 11:03:04.668161 13203 mvcc.cc:205] Tried to move back new op lower bound from 7334624397985947648 to 7334624397903462400. Current Snapshot: MvccSnapshot[applied={T|T < 7334624397985947648}]
I20260929 11:03:04.670449 12713 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45618:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 11:03:04.670542 12713 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45618:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 11:03:04.671299 12713 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 74ee9cec78714b25adc4cd99aaf5956e: Sending DeleteTablet for 3 replicas of tablet 42e11d8cc46f41ac8b859ad7f259c559
I20260929 11:03:04.671633 13106 tablet_service.cc:1514] Processing DeleteTablet for tablet 42e11d8cc46f41ac8b859ad7f259c559 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:03:04 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:33864
I20260929 11:03:04.671643 12972 tablet_service.cc:1514] Processing DeleteTablet for tablet 42e11d8cc46f41ac8b859ad7f259c559 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:03:04 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:48232
I20260929 11:03:04.671633 12838 tablet_service.cc:1514] Processing DeleteTablet for tablet 42e11d8cc46f41ac8b859ad7f259c559 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:03:04 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:39790
I20260929 11:03:04.671988 13212 tablet_replica.cc:330] T 42e11d8cc46f41ac8b859ad7f259c559 P 7e54a9510ff14895a4d054d30070b6d1: stopping tablet replica
I20260929 11:03:04.672250 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 12775
I20260929 11:03:04.672976 13213 tablet_replica.cc:330] T 42e11d8cc46f41ac8b859ad7f259c559 P 7242a9321b844797b0cd9f9e7a5921a7: stopping tablet replica
I20260929 11:03:04.673074 13213 raft_consensus.cc:2229] T 42e11d8cc46f41ac8b859ad7f259c559 P 7242a9321b844797b0cd9f9e7a5921a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 11:03:04.673167 13213 raft_consensus.cc:2258] T 42e11d8cc46f41ac8b859ad7f259c559 P 7242a9321b844797b0cd9f9e7a5921a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 11:03:04.674374 13213 ts_tablet_manager.cc:1905] T 42e11d8cc46f41ac8b859ad7f259c559 P 7242a9321b844797b0cd9f9e7a5921a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 11:03:04.676373 13213 ts_tablet_manager.cc:1918] T 42e11d8cc46f41ac8b859ad7f259c559 P 7242a9321b844797b0cd9f9e7a5921a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 11:03:04.676451 13213 log.cc:1196] T 42e11d8cc46f41ac8b859ad7f259c559 P 7242a9321b844797b0cd9f9e7a5921a7: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790679549986039-27107-0/minicluster-data/ts-1/wal/wals/42e11d8cc46f41ac8b859ad7f259c559
I20260929 11:03:04.676736 13213 ts_tablet_manager.cc:1939] T 42e11d8cc46f41ac8b859ad7f259c559 P 7242a9321b844797b0cd9f9e7a5921a7: Deleting consensus metadata
I20260929 11:03:04.677127 12697 catalog_manager.cc:4841] TS 7242a9321b844797b0cd9f9e7a5921a7 (127.26.120.194:38953): tablet 42e11d8cc46f41ac8b859ad7f259c559 (table test-table [id=8a812701897e4389ac34374af263310e]) successfully deleted
W20260929 11:03:04.677925 12698 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.120.193:44199 (error 108)
I20260929 11:03:04.678153 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 12909
I20260929 11:03:04.678773 13184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:04.678294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:44199 (local address 127.0.0.1:39804)
0929 11:03:04.678691 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:04.678731 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.193:44199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365}
W20260929 11:03:04.678948 12698 catalog_manager.cc:4584] TS 7e54a9510ff14895a4d054d30070b6d1 (127.26.120.193:44199): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 42e11d8cc46f41ac8b859ad7f259c559: Network error: Client connection negotiation failed: client connection to 127.26.120.193:44199: connect: Connection refused (error 111)
I20260929 11:03:04.683076 13214 tablet_replica.cc:330] T 42e11d8cc46f41ac8b859ad7f259c559 P 429a7629064f450b9d31429705424332: stopping tablet replica
I20260929 11:03:04.683247 13214 raft_consensus.cc:2229] T 42e11d8cc46f41ac8b859ad7f259c559 P 429a7629064f450b9d31429705424332 [term 1 LEADER]: Raft consensus shutting down.
I20260929 11:03:04.683423 13214 raft_consensus.cc:2258] T 42e11d8cc46f41ac8b859ad7f259c559 P 429a7629064f450b9d31429705424332 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 11:03:04.683723 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 13043
I20260929 11:03:04.683869 13214 ts_tablet_manager.cc:1905] T 42e11d8cc46f41ac8b859ad7f259c559 P 429a7629064f450b9d31429705424332: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 11:03:04.688041 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 12681
I20260929 11:03:04.688078 13184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:04.687950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:37455 (local address 127.0.0.1:33886)
0929 11:03:04.688009 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:04.688041 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.195:37455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
2026-09-29T11:03:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2112 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-29T11:03:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T11:03:04Z Disabled control of system clock
I20260929 11:03:04.717208 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:44283
--webserver_interface=127.26.120.254
--webserver_port=0
--builtin_ntp_servers=127.26.120.212:41035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:44283
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 11:03:04.797953 13219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:04.798202 13219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:04.798223 13219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:04.799345 13219 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:03:04.799383 13219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:04.799396 13219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:04.799409 13219 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:03:04.799422 13219 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:03:04.800746 13219 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:41035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:44283
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:44283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:04.800936 13219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:04.801262 13219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:04.806298 13225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:04.806294 13227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:04.806294 13224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:04.806447 13219 server_base.cc:1034] running on GCE node
I20260929 11:03:04.806804 13219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:04.807119 13219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:04.808298 13219 hybrid_clock.cc:648] HybridClock initialized: now 1790679784808303 us; error 52 us; skew 500 ppm
I20260929 11:03:04.809760 13219 webserver.cc:460] Webserver started at http://127.26.120.254:37653/ using document root <none> and password file <none>
I20260929 11:03:04.810045 13219 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:04.810170 13219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:04.810333 13219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:04.811487 13219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "1a646b041e6748bab77646f4355d1317"
format_stamp: "Formatted at 2026-09-29 11:03:04 on dist-test-slave-2lzq"
server_key: "15af06ec9675602e0ddfcb996f83b39a"
server_key_iv: "9f4546e565c8c5e370b2b98f6a21ace3"
server_key_version: "encryptionkey@0"
I20260929 11:03:04.811860 13219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "1a646b041e6748bab77646f4355d1317"
format_stamp: "Formatted at 2026-09-29 11:03:04 on dist-test-slave-2lzq"
server_key: "15af06ec9675602e0ddfcb996f83b39a"
server_key_iv: "9f4546e565c8c5e370b2b98f6a21ace3"
server_key_version: "encryptionkey@0"
I20260929 11:03:04.813246 13219 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 11:03:04.814042 13233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:04.814285 13219 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 11:03:04.814407 13219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "1a646b041e6748bab77646f4355d1317"
format_stamp: "Formatted at 2026-09-29 11:03:04 on dist-test-slave-2lzq"
server_key: "15af06ec9675602e0ddfcb996f83b39a"
server_key_iv: "9f4546e565c8c5e370b2b98f6a21ace3"
server_key_version: "encryptionkey@0"
I20260929 11:03:04.814489 13219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:04.838521 13219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:04.838855 13219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:04.839010 13219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:04.843099 13219 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:44283
I20260929 11:03:04.843152 13285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:44283 every 8 connection(s)
I20260929 11:03:04.843520 13219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:03:04.844507 13286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:04.846836 13286 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a646b041e6748bab77646f4355d1317: Bootstrap starting.
I20260929 11:03:04.847464 13286 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a646b041e6748bab77646f4355d1317: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:04.847780 13286 log.cc:824] T 00000000000000000000000000000000 P 1a646b041e6748bab77646f4355d1317: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:04.848471 13286 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a646b041e6748bab77646f4355d1317: No bootstrap required, opened a new log
I20260929 11:03:04.850047 13286 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a646b041e6748bab77646f4355d1317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a646b041e6748bab77646f4355d1317" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44283 } }
I20260929 11:03:04.850265 13286 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a646b041e6748bab77646f4355d1317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:04.850288 13286 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a646b041e6748bab77646f4355d1317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a646b041e6748bab77646f4355d1317, State: Initialized, Role: FOLLOWER
I20260929 11:03:04.850458 13286 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a646b041e6748bab77646f4355d1317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a646b041e6748bab77646f4355d1317" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44283 } }
I20260929 11:03:04.850519 13286 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a646b041e6748bab77646f4355d1317 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 11:03:04.850560 13286 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a646b041e6748bab77646f4355d1317 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 11:03:04.850610 13286 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a646b041e6748bab77646f4355d1317 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:04.851126 13286 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a646b041e6748bab77646f4355d1317 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a646b041e6748bab77646f4355d1317" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44283 } }
I20260929 11:03:04.851248 13286 leader_election.cc:302] T 00000000000000000000000000000000 P 1a646b041e6748bab77646f4355d1317 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a646b041e6748bab77646f4355d1317; no voters:
I20260929 11:03:04.851431 13286 leader_election.cc:288] T 00000000000000000000000000000000 P 1a646b041e6748bab77646f4355d1317 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 11:03:04.851521 13289 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a646b041e6748bab77646f4355d1317 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:03:04.851732 13286 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a646b041e6748bab77646f4355d1317 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:03:04.851716 13289 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a646b041e6748bab77646f4355d1317 [term 1 LEADER]: Becoming Leader. State: Replica: 1a646b041e6748bab77646f4355d1317, State: Running, Role: LEADER
I20260929 11:03:04.851943 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 13219
I20260929 11:03:04.851929 13289 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a646b041e6748bab77646f4355d1317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a646b041e6748bab77646f4355d1317" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44283 } }
I20260929 11:03:04.852056 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 11:03:04.852270 27107 external_mini_cluster.cc:1428] Setting key 3f852cc6bc5f4a0427f5e1b345a999b0
I20260929 11:03:04.852525 13291 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a646b041e6748bab77646f4355d1317 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a646b041e6748bab77646f4355d1317. Latest consensus state: current_term: 1 leader_uuid: "1a646b041e6748bab77646f4355d1317" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a646b041e6748bab77646f4355d1317" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44283 } } }
I20260929 11:03:04.852615 13291 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a646b041e6748bab77646f4355d1317 [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:04.852926 13290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a646b041e6748bab77646f4355d1317 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a646b041e6748bab77646f4355d1317" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a646b041e6748bab77646f4355d1317" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 44283 } } }
I20260929 11:03:04.853006 13290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a646b041e6748bab77646f4355d1317 [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:04.852993 13298 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:03:04.853560 13298 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:03:04.855187 13298 catalog_manager.cc:1347] Generated new cluster ID: 62f079dc58b745ee8b59b91ddb473aaf
I20260929 11:03:04.855242 13298 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 11:03:04.858199 13304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:04.852990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44166 (local address 127.26.120.254:44283)
0929 11:03:04.853307 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:04.853316 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:04.853542 (+ 226us) server_negotiation.cc:407] Connection header received
0929 11:03:04.853677 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:04.853685 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:04.853772 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:04.853930 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:04.854290 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.855390 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.856196 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:04.856626 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:04.856759 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:04.857485 (+ 726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:04.857488 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:04.857489 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:04.857664 (+ 175us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:04.857696 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:04.857706 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:04.857780 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:04.857853 (+ 73us) server_negotiation.cc:299] Negotiation successful
0929 11:03:04.857943 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":87,"threads_started":1}
I20260929 11:03:04.870273 13298 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 11:03:04.870934 13298 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:03:04.884795 13298 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a646b041e6748bab77646f4355d1317: Generated new TSK 0
I20260929 11:03:04.885036 13298 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 11:03:04.886350 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.193:0
--local_ip_for_outbound_sockets=127.26.120.193
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:44283
--builtin_ntp_servers=127.26.120.212:41035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 11:03:05.003993 13310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:05.004272 13310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:05.004316 13310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:05.006016 13310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:05.006112 13310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:05.006158 13310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.193
I20260929 11:03:05.008195 13310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:41035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:44283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.193
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:05.008462 13310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:05.008823 13310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:05.014211 13318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:05.014245 13315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:05.014211 13316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:05.014520 13310 server_base.cc:1034] running on GCE node
I20260929 11:03:05.014731 13310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:05.014976 13310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:05.016139 13310 hybrid_clock.cc:648] HybridClock initialized: now 1790679785016102 us; error 52 us; skew 500 ppm
I20260929 11:03:05.017567 13310 webserver.cc:460] Webserver started at http://127.26.120.193:41863/ using document root <none> and password file <none>
I20260929 11:03:05.017859 13310 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:05.017931 13310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:05.018115 13310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:05.019268 13310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-0/data/instance:
uuid: "fc12e93b549c45b7ae17b7369ecca170"
format_stamp: "Formatted at 2026-09-29 11:03:05 on dist-test-slave-2lzq"
server_key: "0a68e8e9fb96973e941b4352808a584f"
server_key_iv: "85649e9dd3f7c0a839ba143789e5d3a9"
server_key_version: "encryptionkey@0"
I20260929 11:03:05.019644 13310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance:
uuid: "fc12e93b549c45b7ae17b7369ecca170"
format_stamp: "Formatted at 2026-09-29 11:03:05 on dist-test-slave-2lzq"
server_key: "0a68e8e9fb96973e941b4352808a584f"
server_key_iv: "85649e9dd3f7c0a839ba143789e5d3a9"
server_key_version: "encryptionkey@0"
I20260929 11:03:05.020963 13310 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 11:03:05.021808 13324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:05.021970 13310 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:03:05.022081 13310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-0/wal
uuid: "fc12e93b549c45b7ae17b7369ecca170"
format_stamp: "Formatted at 2026-09-29 11:03:05 on dist-test-slave-2lzq"
server_key: "0a68e8e9fb96973e941b4352808a584f"
server_key_iv: "85649e9dd3f7c0a839ba143789e5d3a9"
server_key_version: "encryptionkey@0"
I20260929 11:03:05.022159 13310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:05.034313 13310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:05.034616 13310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:05.034765 13310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:05.034999 13310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:03:05.035331 13310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:03:05.035364 13310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:05.035408 13310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:03:05.035434 13310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:05.041373 13310 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.193:32935
I20260929 11:03:05.041429 13437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.193:32935 every 8 connection(s)
I20260929 11:03:05.041742 13310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
I20260929 11:03:05.042207 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 13310
I20260929 11:03:05.042330 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance
I20260929 11:03:05.042577 27107 external_mini_cluster.cc:1428] Setting key 2042c2c3d1bcbd14be316978aaa07265
I20260929 11:03:05.043603 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.194:0
--local_ip_for_outbound_sockets=127.26.120.194
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:44283
--builtin_ntp_servers=127.26.120.212:41035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 11:03:05.047551 13304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:05.042522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:33167 (local address 127.26.120.254:44283)
0929 11:03:05.042574 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:05.042577 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:05.042924 (+ 347us) server_negotiation.cc:407] Connection header received
0929 11:03:05.043514 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:05.043515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:05.043524 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:05.043564 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:05.044454 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:05.044927 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:05.045941 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:05.046229 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:05.046314 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:05.047289 (+ 975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:05.047293 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:05.047293 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:05.047298 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:05.047312 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:05.047316 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:05.047370 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:05.047446 (+ 76us) server_negotiation.cc:299] Negotiation successful
0929 11:03:05.047475 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 11:03:05.047773 13440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:05.042539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44283 (local address 127.26.120.193:33167)
0929 11:03:05.042787 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:05.042833 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:05.043408 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:05.043640 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:05.043645 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:05.043657 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:03:05.044362 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:05.044369 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:05.045031 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:05.045033 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:05.045823 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:05.045826 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:05.046343 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:05.046344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:05.046500 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:05.047125 (+ 625us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 11:03:05.047125 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:03:05.047163 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 11:03:05.047186 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:03:05.047370 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:03:05.047376 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:03:05.047379 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 11:03:05.047449 (+ 70us) client_negotiation.cc:240] Negotiation successful
0929 11:03:05.047549 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":89,"threads_started":1}
I20260929 11:03:05.048298 13438 heartbeater.cc:347] Connected to a master server at 127.26.120.254:44283
I20260929 11:03:05.048381 13438 heartbeater.cc:464] Registering TS with master...
I20260929 11:03:05.048579 13438 heartbeater.cc:511] Master 127.26.120.254:44283 requested a full tablet report, sending...
I20260929 11:03:05.049100 13250 ts_manager.cc:194] Registered new tserver with Master: fc12e93b549c45b7ae17b7369ecca170 (127.26.120.193:32935)
I20260929 11:03:05.049721 13250 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.120.193:33167
W20260929 11:03:05.126010 13441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:05.126286 13441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:05.126325 13441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:05.127481 13441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:05.127547 13441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:05.127591 13441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.194
I20260929 11:03:05.128980 13441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:41035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:44283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.194
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:05.129251 13441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:05.129562 13441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 11:03:05.134759 13441 server_base.cc:1034] running on GCE node
W20260929 11:03:05.134742 13447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:05.134723 13446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:05.134975 13449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:05.135221 13441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:05.135511 13441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:05.136670 13441 hybrid_clock.cc:648] HybridClock initialized: now 1790679785136661 us; error 27 us; skew 500 ppm
I20260929 11:03:05.137954 13441 webserver.cc:460] Webserver started at http://127.26.120.194:37163/ using document root <none> and password file <none>
I20260929 11:03:05.138219 13441 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:05.138267 13441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:05.138434 13441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:05.139529 13441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-1/data/instance:
uuid: "d988027320354465b79936bb43b5a4ce"
format_stamp: "Formatted at 2026-09-29 11:03:05 on dist-test-slave-2lzq"
server_key: "ea1af49f6f92c0e137977eee0d97ceae"
server_key_iv: "4d6ffabc0d166d37c3f462254371c710"
server_key_version: "encryptionkey@0"
I20260929 11:03:05.139883 13441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance:
uuid: "d988027320354465b79936bb43b5a4ce"
format_stamp: "Formatted at 2026-09-29 11:03:05 on dist-test-slave-2lzq"
server_key: "ea1af49f6f92c0e137977eee0d97ceae"
server_key_iv: "4d6ffabc0d166d37c3f462254371c710"
server_key_version: "encryptionkey@0"
I20260929 11:03:05.141172 13441 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 11:03:05.141904 13455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:05.142112 13441 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 11:03:05.142210 13441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-1/wal
uuid: "d988027320354465b79936bb43b5a4ce"
format_stamp: "Formatted at 2026-09-29 11:03:05 on dist-test-slave-2lzq"
server_key: "ea1af49f6f92c0e137977eee0d97ceae"
server_key_iv: "4d6ffabc0d166d37c3f462254371c710"
server_key_version: "encryptionkey@0"
I20260929 11:03:05.142294 13441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:05.164620 13441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:05.164964 13441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:05.165124 13441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:05.165380 13441 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:03:05.165740 13441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:03:05.165796 13441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:05.165843 13441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:03:05.165866 13441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:05.171967 13441 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.194:40851
I20260929 11:03:05.172029 13568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.194:40851 every 8 connection(s)
I20260929 11:03:05.172411 13441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
I20260929 11:03:05.177966 13304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:05.173219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:44427 (local address 127.26.120.254:44283)
0929 11:03:05.173339 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:05.173341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:05.173574 (+ 233us) server_negotiation.cc:407] Connection header received
0929 11:03:05.174161 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:05.174162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:05.174175 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:05.174224 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:05.174943 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:05.175454 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:05.176474 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:05.176679 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:05.176754 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:05.177651 (+ 897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:05.177653 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:05.177653 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:05.177658 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:05.177680 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:05.177686 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:05.177747 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:05.177853 (+ 106us) server_negotiation.cc:299] Negotiation successful
0929 11:03:05.177884 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 11:03:05.178221 13571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:05.173201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44283 (local address 127.26.120.194:44427)
0929 11:03:05.173443 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:05.173478 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:05.174012 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:05.174321 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:05.174325 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:05.174341 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:03:05.174841 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:05.174845 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:05.175572 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:05.175573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:05.176378 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:05.176380 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:05.176778 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:05.176779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:05.176913 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:05.177505 (+ 592us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 11:03:05.177505 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:03:05.177534 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 11:03:05.177557 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:03:05.177759 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:03:05.177766 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:03:05.177769 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 11:03:05.177854 (+ 85us) client_negotiation.cc:240] Negotiation successful
0929 11:03:05.177957 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":98,"threads_started":1}
I20260929 11:03:05.178777 13569 heartbeater.cc:347] Connected to a master server at 127.26.120.254:44283
I20260929 11:03:05.178874 13569 heartbeater.cc:464] Registering TS with master...
I20260929 11:03:05.179116 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 13441
I20260929 11:03:05.179126 13569 heartbeater.cc:511] Master 127.26.120.254:44283 requested a full tablet report, sending...
I20260929 11:03:05.179234 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance
I20260929 11:03:05.179443 27107 external_mini_cluster.cc:1428] Setting key c030deb545b8eacb1dbd54c427bde484
I20260929 11:03:05.179590 13250 ts_manager.cc:194] Registered new tserver with Master: d988027320354465b79936bb43b5a4ce (127.26.120.194:40851)
I20260929 11:03:05.180334 13250 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.120.194:44427
I20260929 11:03:05.180620 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.195:0
--local_ip_for_outbound_sockets=127.26.120.195
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:44283
--builtin_ntp_servers=127.26.120.212:41035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 11:03:05.261312 13572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:05.261510 13572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:05.261533 13572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:05.262713 13572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:05.262755 13572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:05.262780 13572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.195
I20260929 11:03:05.264132 13572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:41035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:44283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.195
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:05.264380 13572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:05.264689 13572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:05.269599 13578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:05.269600 13580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:05.269626 13577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:05.269896 13572 server_base.cc:1034] running on GCE node
I20260929 11:03:05.270197 13572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:05.270484 13572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:05.271642 13572 hybrid_clock.cc:648] HybridClock initialized: now 1790679785271638 us; error 36 us; skew 500 ppm
I20260929 11:03:05.273008 13572 webserver.cc:460] Webserver started at http://127.26.120.195:39093/ using document root <none> and password file <none>
I20260929 11:03:05.273288 13572 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:05.273365 13572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:05.273522 13572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:05.274657 13572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-2/data/instance:
uuid: "af7611f4cfac4df6b1758c37d4ce6051"
format_stamp: "Formatted at 2026-09-29 11:03:05 on dist-test-slave-2lzq"
server_key: "42d8edf9d960919ad9427bbcc5daef3c"
server_key_iv: "d478638b8f8694cdb2e00a26a863efe2"
server_key_version: "encryptionkey@0"
I20260929 11:03:05.275024 13572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance:
uuid: "af7611f4cfac4df6b1758c37d4ce6051"
format_stamp: "Formatted at 2026-09-29 11:03:05 on dist-test-slave-2lzq"
server_key: "42d8edf9d960919ad9427bbcc5daef3c"
server_key_iv: "d478638b8f8694cdb2e00a26a863efe2"
server_key_version: "encryptionkey@0"
I20260929 11:03:05.276279 13572 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 11:03:05.276990 13586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:05.277163 13572 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 11:03:05.277269 13572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-2/wal
uuid: "af7611f4cfac4df6b1758c37d4ce6051"
format_stamp: "Formatted at 2026-09-29 11:03:05 on dist-test-slave-2lzq"
server_key: "42d8edf9d960919ad9427bbcc5daef3c"
server_key_iv: "d478638b8f8694cdb2e00a26a863efe2"
server_key_version: "encryptionkey@0"
I20260929 11:03:05.277352 13572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:05.287864 13572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:05.288161 13572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:05.288312 13572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:05.288542 13572 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:03:05.288887 13572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:03:05.288949 13572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:05.288990 13572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:03:05.289021 13572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:05.294934 13572 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.195:45899
I20260929 11:03:05.294976 13699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.195:45899 every 8 connection(s)
I20260929 11:03:05.295295 13572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
I20260929 11:03:05.300539 13304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:05.296015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:51093 (local address 127.26.120.254:44283)
0929 11:03:05.296072 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:05.296074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:05.296332 (+ 258us) server_negotiation.cc:407] Connection header received
0929 11:03:05.296787 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:05.296788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:05.296802 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:05.296862 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:05.297522 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:05.298006 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:05.299059 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:05.299280 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:05.299385 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:05.300257 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:05.300258 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:05.300258 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:05.300263 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:05.300279 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:05.300283 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:05.300332 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:05.300426 (+ 94us) server_negotiation.cc:299] Negotiation successful
0929 11:03:05.300458 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260929 11:03:05.300781 13702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:05.295986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:44283 (local address 127.26.120.195:51093)
0929 11:03:05.296199 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:05.296240 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:05.296706 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:05.296934 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:05.296937 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:05.296946 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:03:05.297447 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:05.297451 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:05.298148 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:05.298149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:05.298959 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:05.298962 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:05.299407 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:05.299409 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:05.299537 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:05.300108 (+ 571us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 11:03:05.300108 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:03:05.300136 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 11:03:05.300159 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:03:05.300333 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:03:05.300338 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:03:05.300341 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 11:03:05.300428 (+ 87us) client_negotiation.cc:240] Negotiation successful
0929 11:03:05.300527 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":106,"threads_started":1}
I20260929 11:03:05.301327 13700 heartbeater.cc:347] Connected to a master server at 127.26.120.254:44283
I20260929 11:03:05.301426 13700 heartbeater.cc:464] Registering TS with master...
I20260929 11:03:05.301563 13700 heartbeater.cc:511] Master 127.26.120.254:44283 requested a full tablet report, sending...
I20260929 11:03:05.301934 13250 ts_manager.cc:194] Registered new tserver with Master: af7611f4cfac4df6b1758c37d4ce6051 (127.26.120.195:45899)
I20260929 11:03:05.302330 13250 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.120.195:51093
I20260929 11:03:05.305487 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 13572
I20260929 11:03:05.305573 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance
I20260929 11:03:05.305786 27107 external_mini_cluster.cc:1428] Setting key 68f2c7d3f34abbb0f3685196eff0c516
I20260929 11:03:05.306905 27107 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 11:03:05.311031 13304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:05.308248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37408 (local address 127.26.120.254:44283)
0929 11:03:05.308290 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:05.308291 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:05.308430 (+ 139us) server_negotiation.cc:407] Connection header received
0929 11:03:05.308483 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:05.308483 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:05.308492 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:05.308533 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:05.308685 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:05.309144 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:05.309855 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:05.310037 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:05.310136 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:05.310801 (+ 665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:05.310802 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:05.310802 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:05.310806 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:05.310816 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:05.310819 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:05.310890 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:05.310971 (+ 81us) server_negotiation.cc:299] Negotiation successful
0929 11:03:05.310980 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 11:03:05.312412 13250 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37408:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 11:03:05.313866 13250 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 11:03:05.320271 13708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:05.315908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:32935 (local address 127.0.0.1:42536)
0929 11:03:05.316277 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:05.316318 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:05.316403 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:05.316676 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:05.316679 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:05.316681 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:03:05.316761 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:05.316763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:05.318152 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:05.318153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:05.319091 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:05.319094 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:05.319990 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:05.319991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:05.320046 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:05.320057 (+ 11us) client_negotiation.cc:769] Sending connection context
0929 11:03:05.320137 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 11:03:05.320187 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":9,"thread_start_us":75,"threads_started":1}
I20260929 11:03:05.320286 13711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:05.316022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42536 (local address 127.26.120.193:32935)
0929 11:03:05.316218 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:05.316223 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:05.316381 (+ 158us) server_negotiation.cc:407] Connection header received
0929 11:03:05.316486 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:05.316488 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:05.316505 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:05.316602 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:03:05.316892 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:05.318036 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:05.319179 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:05.319905 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:05.319988 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:05.320079 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:05.320155 (+ 76us) server_negotiation.cc:299] Negotiation successful
0929 11:03:05.320194 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":110,"threads_started":1}
I20260929 11:03:05.320585 13712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:05.316132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52128 (local address 127.26.120.195:45899)
0929 11:03:05.316305 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:05.316308 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:05.316647 (+ 339us) server_negotiation.cc:407] Connection header received
0929 11:03:05.316701 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:05.316703 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:05.316718 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:05.316789 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:03:05.316977 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:05.317746 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:05.319376 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:05.320059 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:05.320130 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:05.320205 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:05.320470 (+ 265us) server_negotiation.cc:299] Negotiation successful
0929 11:03:05.320510 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":81,"threads_started":1}
I20260929 11:03:05.320411 13709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:05.315946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:45899 (local address 127.0.0.1:52128)
0929 11:03:05.316511 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:05.316527 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:05.316587 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:05.316853 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:05.316855 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:05.316857 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:03:05.316909 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:05.316911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:05.317831 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:05.317832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:05.319279 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:05.319281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:05.320134 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:05.320134 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:05.320186 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:05.320189 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 11:03:05.320289 (+ 100us) client_negotiation.cc:240] Negotiation successful
0929 11:03:05.320331 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"mutex_wait_us":24,"thread_start_us":65,"threads_started":1}
I20260929 11:03:05.321314 13710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:05.315927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:40851 (local address 127.0.0.1:33950)
0929 11:03:05.316359 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:05.316369 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:05.316419 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:05.316676 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:05.316679 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:05.316684 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:03:05.316761 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:05.316764 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:05.318129 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:05.318130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:05.319279 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:05.319281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:05.321075 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:05.321076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:05.321139 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:05.321143 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 11:03:05.321202 (+ 59us) client_negotiation.cc:240] Negotiation successful
0929 11:03:05.321240 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"mutex_wait_us":7,"thread_start_us":50,"threads_started":1}
I20260929 11:03:05.321367 13372 tablet_service.cc:1467] Processing CreateTablet for tablet 656dad5e4bec41e48271c2aab461bd3b (DEFAULT_TABLE table=test-table [id=6a0c6f11bf484651a49aa8cc2e80bac0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:03:05.321543 13713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:05.316167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33950 (local address 127.26.120.194:40851)
0929 11:03:05.316345 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:05.316348 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:05.316411 (+ 63us) server_negotiation.cc:407] Connection header received
0929 11:03:05.316486 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:05.316488 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:05.316505 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:05.316592 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:03:05.316845 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:05.318036 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:05.320252 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:05.320926 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:05.321024 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:05.321111 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:05.321417 (+ 306us) server_negotiation.cc:299] Negotiation successful
0929 11:03:05.321462 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":100,"threads_started":1}
I20260929 11:03:05.321748 13372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 656dad5e4bec41e48271c2aab461bd3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:05.322737 13634 tablet_service.cc:1467] Processing CreateTablet for tablet 656dad5e4bec41e48271c2aab461bd3b (DEFAULT_TABLE table=test-table [id=6a0c6f11bf484651a49aa8cc2e80bac0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:03:05.323043 13634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 656dad5e4bec41e48271c2aab461bd3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:05.324767 13714 tablet_bootstrap.cc:492] T 656dad5e4bec41e48271c2aab461bd3b P fc12e93b549c45b7ae17b7369ecca170: Bootstrap starting.
I20260929 11:03:05.325134 13503 tablet_service.cc:1467] Processing CreateTablet for tablet 656dad5e4bec41e48271c2aab461bd3b (DEFAULT_TABLE table=test-table [id=6a0c6f11bf484651a49aa8cc2e80bac0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:03:05.325490 13503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 656dad5e4bec41e48271c2aab461bd3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:05.325505 13714 tablet_bootstrap.cc:654] T 656dad5e4bec41e48271c2aab461bd3b P fc12e93b549c45b7ae17b7369ecca170: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:05.325840 13714 log.cc:824] T 656dad5e4bec41e48271c2aab461bd3b P fc12e93b549c45b7ae17b7369ecca170: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:05.326154 13715 tablet_bootstrap.cc:492] T 656dad5e4bec41e48271c2aab461bd3b P af7611f4cfac4df6b1758c37d4ce6051: Bootstrap starting.
I20260929 11:03:05.326706 13714 tablet_bootstrap.cc:492] T 656dad5e4bec41e48271c2aab461bd3b P fc12e93b549c45b7ae17b7369ecca170: No bootstrap required, opened a new log
I20260929 11:03:05.326793 13714 ts_tablet_manager.cc:1397] T 656dad5e4bec41e48271c2aab461bd3b P fc12e93b549c45b7ae17b7369ecca170: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260929 11:03:05.326833 13715 tablet_bootstrap.cc:654] T 656dad5e4bec41e48271c2aab461bd3b P af7611f4cfac4df6b1758c37d4ce6051: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:05.327158 13715 log.cc:824] T 656dad5e4bec41e48271c2aab461bd3b P af7611f4cfac4df6b1758c37d4ce6051: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:05.328052 13715 tablet_bootstrap.cc:492] T 656dad5e4bec41e48271c2aab461bd3b P af7611f4cfac4df6b1758c37d4ce6051: No bootstrap required, opened a new log
I20260929 11:03:05.328158 13715 ts_tablet_manager.cc:1397] T 656dad5e4bec41e48271c2aab461bd3b P af7611f4cfac4df6b1758c37d4ce6051: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260929 11:03:05.328755 13718 tablet_bootstrap.cc:492] T 656dad5e4bec41e48271c2aab461bd3b P d988027320354465b79936bb43b5a4ce: Bootstrap starting.
I20260929 11:03:05.329283 13718 tablet_bootstrap.cc:654] T 656dad5e4bec41e48271c2aab461bd3b P d988027320354465b79936bb43b5a4ce: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:05.329411 13714 raft_consensus.cc:348] T 656dad5e4bec41e48271c2aab461bd3b P fc12e93b549c45b7ae17b7369ecca170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc12e93b549c45b7ae17b7369ecca170" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 32935 } } peers { permanent_uuid: "d988027320354465b79936bb43b5a4ce" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 40851 } } peers { permanent_uuid: "af7611f4cfac4df6b1758c37d4ce6051" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45899 } }
I20260929 11:03:05.329588 13714 raft_consensus.cc:374] T 656dad5e4bec41e48271c2aab461bd3b P fc12e93b549c45b7ae17b7369ecca170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:05.329567 13718 log.cc:824] T 656dad5e4bec41e48271c2aab461bd3b P d988027320354465b79936bb43b5a4ce: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:05.329634 13714 raft_consensus.cc:729] T 656dad5e4bec41e48271c2aab461bd3b P fc12e93b549c45b7ae17b7369ecca170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc12e93b549c45b7ae17b7369ecca170, State: Initialized, Role: FOLLOWER
I20260929 11:03:05.329780 13714 consensus_queue.cc:260] T 656dad5e4bec41e48271c2aab461bd3b P fc12e93b549c45b7ae17b7369ecca170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc12e93b549c45b7ae17b7369ecca170" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 32935 } } peers { permanent_uuid: "d988027320354465b79936bb43b5a4ce" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 40851 } } peers { permanent_uuid: "af7611f4cfac4df6b1758c37d4ce6051" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45899 } }
I20260929 11:03:05.330045 13714 ts_tablet_manager.cc:1428] T 656dad5e4bec41e48271c2aab461bd3b P fc12e93b549c45b7ae17b7369ecca170: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260929 11:03:05.330195 13438 heartbeater.cc:503] Master 127.26.120.254:44283 was elected leader, sending a full tablet report...
I20260929 11:03:05.330374 13718 tablet_bootstrap.cc:492] T 656dad5e4bec41e48271c2aab461bd3b P d988027320354465b79936bb43b5a4ce: No bootstrap required, opened a new log
I20260929 11:03:05.330454 13718 ts_tablet_manager.cc:1397] T 656dad5e4bec41e48271c2aab461bd3b P d988027320354465b79936bb43b5a4ce: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 11:03:05.330821 13715 raft_consensus.cc:348] T 656dad5e4bec41e48271c2aab461bd3b P af7611f4cfac4df6b1758c37d4ce6051 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc12e93b549c45b7ae17b7369ecca170" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 32935 } } peers { permanent_uuid: "d988027320354465b79936bb43b5a4ce" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 40851 } } peers { permanent_uuid: "af7611f4cfac4df6b1758c37d4ce6051" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45899 } }
I20260929 11:03:05.330971 13715 raft_consensus.cc:374] T 656dad5e4bec41e48271c2aab461bd3b P af7611f4cfac4df6b1758c37d4ce6051 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:05.331009 13715 raft_consensus.cc:729] T 656dad5e4bec41e48271c2aab461bd3b P af7611f4cfac4df6b1758c37d4ce6051 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af7611f4cfac4df6b1758c37d4ce6051, State: Initialized, Role: FOLLOWER
I20260929 11:03:05.331120 13715 consensus_queue.cc:260] T 656dad5e4bec41e48271c2aab461bd3b P af7611f4cfac4df6b1758c37d4ce6051 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc12e93b549c45b7ae17b7369ecca170" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 32935 } } peers { permanent_uuid: "d988027320354465b79936bb43b5a4ce" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 40851 } } peers { permanent_uuid: "af7611f4cfac4df6b1758c37d4ce6051" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45899 } }
I20260929 11:03:05.331373 13715 ts_tablet_manager.cc:1428] T 656dad5e4bec41e48271c2aab461bd3b P af7611f4cfac4df6b1758c37d4ce6051: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260929 11:03:05.331491 13700 heartbeater.cc:503] Master 127.26.120.254:44283 was elected leader, sending a full tablet report...
I20260929 11:03:05.332801 13718 raft_consensus.cc:348] T 656dad5e4bec41e48271c2aab461bd3b P d988027320354465b79936bb43b5a4ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc12e93b549c45b7ae17b7369ecca170" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 32935 } } peers { permanent_uuid: "d988027320354465b79936bb43b5a4ce" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 40851 } } peers { permanent_uuid: "af7611f4cfac4df6b1758c37d4ce6051" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45899 } }
I20260929 11:03:05.332921 13718 raft_consensus.cc:374] T 656dad5e4bec41e48271c2aab461bd3b P d988027320354465b79936bb43b5a4ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:05.332942 13718 raft_consensus.cc:729] T 656dad5e4bec41e48271c2aab461bd3b P d988027320354465b79936bb43b5a4ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d988027320354465b79936bb43b5a4ce, State: Initialized, Role: FOLLOWER
I20260929 11:03:05.333040 13718 consensus_queue.cc:260] T 656dad5e4bec41e48271c2aab461bd3b P d988027320354465b79936bb43b5a4ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc12e93b549c45b7ae17b7369ecca170" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 32935 } } peers { permanent_uuid: "d988027320354465b79936bb43b5a4ce" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 40851 } } peers { permanent_uuid: "af7611f4cfac4df6b1758c37d4ce6051" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45899 } }
I20260929 11:03:05.333245 13718 ts_tablet_manager.cc:1428] T 656dad5e4bec41e48271c2aab461bd3b P d988027320354465b79936bb43b5a4ce: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260929 11:03:05.333360 13569 heartbeater.cc:503] Master 127.26.120.254:44283 was elected leader, sending a full tablet report...
W20260929 11:03:05.423019 13570 tablet.cc:2369] T 656dad5e4bec41e48271c2aab461bd3b P d988027320354465b79936bb43b5a4ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 11:03:05.449898 13722 raft_consensus.cc:482] T 656dad5e4bec41e48271c2aab461bd3b P d988027320354465b79936bb43b5a4ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 11:03:05.450114 13722 raft_consensus.cc:504] T 656dad5e4bec41e48271c2aab461bd3b P d988027320354465b79936bb43b5a4ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc12e93b549c45b7ae17b7369ecca170" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 32935 } } peers { permanent_uuid: "d988027320354465b79936bb43b5a4ce" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 40851 } } peers { permanent_uuid: "af7611f4cfac4df6b1758c37d4ce6051" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45899 } }
I20260929 11:03:05.450551 13722 leader_election.cc:288] T 656dad5e4bec41e48271c2aab461bd3b P d988027320354465b79936bb43b5a4ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc12e93b549c45b7ae17b7369ecca170 (127.26.120.193:32935), af7611f4cfac4df6b1758c37d4ce6051 (127.26.120.195:45899)
I20260929 11:03:05.454573 13711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:05.450812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:36017 (local address 127.26.120.193:32935)
0929 11:03:05.450863 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:05.450865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:05.451011 (+ 146us) server_negotiation.cc:407] Connection header received
0929 11:03:05.451076 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:05.451077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:05.451090 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:05.451141 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:03:05.451310 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:05.452034 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:05.453540 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:05.454182 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:05.454265 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:05.454274 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:05.454455 (+ 181us) server_negotiation.cc:299] Negotiation successful
0929 11:03:05.454497 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 11:03:05.454573 13712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:05.450836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:38039 (local address 127.26.120.195:45899)
0929 11:03:05.450885 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:05.450887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:05.450896 (+ 9us) server_negotiation.cc:407] Connection header received
0929 11:03:05.450948 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:05.450949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:05.450959 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:05.451044 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:03:05.451239 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:05.452029 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:05.453546 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:05.454182 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:05.454270 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:05.454277 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:05.454455 (+ 178us) server_negotiation.cc:299] Negotiation successful
0929 11:03:05.454496 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260929 11:03:05.454605 13571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:05.450743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:45899 (local address 127.26.120.194:38039)
0929 11:03:05.450804 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:05.450817 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:05.450878 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:05.451065 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:05.451066 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:05.451068 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:03:05.451163 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:05.451165 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:05.452115 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:05.452116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:05.453437 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:05.453440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:05.454278 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:05.454279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:05.454365 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:05.454369 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 11:03:05.454431 (+ 62us) client_negotiation.cc:240] Negotiation successful
0929 11:03:05.454478 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":34}
I20260929 11:03:05.454613 13723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:05.450784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:32935 (local address 127.26.120.194:36017)
0929 11:03:05.450936 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:05.450945 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:05.450999 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:05.451161 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:05.451162 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:05.451163 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:03:05.451221 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:05.451222 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:05.452115 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:05.452116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:05.453437 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:05.453440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:05.454291 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:05.454291 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:05.454365 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:05.454370 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 11:03:05.454431 (+ 61us) client_negotiation.cc:240] Negotiation successful
0929 11:03:05.454476 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":84,"thread_start_us":99,"threads_started":1}
I20260929 11:03:05.454993 13654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656dad5e4bec41e48271c2aab461bd3b" candidate_uuid: "d988027320354465b79936bb43b5a4ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af7611f4cfac4df6b1758c37d4ce6051" is_pre_election: true
I20260929 11:03:05.454993 13392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656dad5e4bec41e48271c2aab461bd3b" candidate_uuid: "d988027320354465b79936bb43b5a4ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc12e93b549c45b7ae17b7369ecca170" is_pre_election: true
I20260929 11:03:05.455150 13392 raft_consensus.cc:2454] T 656dad5e4bec41e48271c2aab461bd3b P fc12e93b549c45b7ae17b7369ecca170 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d988027320354465b79936bb43b5a4ce in term 0.
I20260929 11:03:05.455150 13654 raft_consensus.cc:2454] T 656dad5e4bec41e48271c2aab461bd3b P af7611f4cfac4df6b1758c37d4ce6051 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d988027320354465b79936bb43b5a4ce in term 0.
I20260929 11:03:05.455482 13459 leader_election.cc:302] T 656dad5e4bec41e48271c2aab461bd3b P d988027320354465b79936bb43b5a4ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d988027320354465b79936bb43b5a4ce, fc12e93b549c45b7ae17b7369ecca170; no voters:
I20260929 11:03:05.455641 13722 raft_consensus.cc:2781] T 656dad5e4bec41e48271c2aab461bd3b P d988027320354465b79936bb43b5a4ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 11:03:05.455745 13722 raft_consensus.cc:482] T 656dad5e4bec41e48271c2aab461bd3b P d988027320354465b79936bb43b5a4ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 11:03:05.455781 13722 raft_consensus.cc:3037] T 656dad5e4bec41e48271c2aab461bd3b P d988027320354465b79936bb43b5a4ce [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:05.456506 13722 raft_consensus.cc:504] T 656dad5e4bec41e48271c2aab461bd3b P d988027320354465b79936bb43b5a4ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc12e93b549c45b7ae17b7369ecca170" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 32935 } } peers { permanent_uuid: "d988027320354465b79936bb43b5a4ce" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 40851 } } peers { permanent_uuid: "af7611f4cfac4df6b1758c37d4ce6051" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45899 } }
I20260929 11:03:05.456657 13722 leader_election.cc:288] T 656dad5e4bec41e48271c2aab461bd3b P d988027320354465b79936bb43b5a4ce [CANDIDATE]: Term 1 election: Requested vote from peers fc12e93b549c45b7ae17b7369ecca170 (127.26.120.193:32935), af7611f4cfac4df6b1758c37d4ce6051 (127.26.120.195:45899)
I20260929 11:03:05.456816 13654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656dad5e4bec41e48271c2aab461bd3b" candidate_uuid: "d988027320354465b79936bb43b5a4ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af7611f4cfac4df6b1758c37d4ce6051"
I20260929 11:03:05.456879 13392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "656dad5e4bec41e48271c2aab461bd3b" candidate_uuid: "d988027320354465b79936bb43b5a4ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc12e93b549c45b7ae17b7369ecca170"
I20260929 11:03:05.456916 13654 raft_consensus.cc:3037] T 656dad5e4bec41e48271c2aab461bd3b P af7611f4cfac4df6b1758c37d4ce6051 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:05.456990 13392 raft_consensus.cc:3037] T 656dad5e4bec41e48271c2aab461bd3b P fc12e93b549c45b7ae17b7369ecca170 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:05.457739 13654 raft_consensus.cc:2454] T 656dad5e4bec41e48271c2aab461bd3b P af7611f4cfac4df6b1758c37d4ce6051 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d988027320354465b79936bb43b5a4ce in term 1.
I20260929 11:03:05.457762 13392 raft_consensus.cc:2454] T 656dad5e4bec41e48271c2aab461bd3b P fc12e93b549c45b7ae17b7369ecca170 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d988027320354465b79936bb43b5a4ce in term 1.
I20260929 11:03:05.457979 13457 leader_election.cc:302] T 656dad5e4bec41e48271c2aab461bd3b P d988027320354465b79936bb43b5a4ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af7611f4cfac4df6b1758c37d4ce6051, d988027320354465b79936bb43b5a4ce; no voters:
I20260929 11:03:05.458102 13722 raft_consensus.cc:2781] T 656dad5e4bec41e48271c2aab461bd3b P d988027320354465b79936bb43b5a4ce [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:03:05.458280 13722 raft_consensus.cc:686] T 656dad5e4bec41e48271c2aab461bd3b P d988027320354465b79936bb43b5a4ce [term 1 LEADER]: Becoming Leader. State: Replica: d988027320354465b79936bb43b5a4ce, State: Running, Role: LEADER
I20260929 11:03:05.458390 13722 consensus_queue.cc:237] T 656dad5e4bec41e48271c2aab461bd3b P d988027320354465b79936bb43b5a4ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc12e93b549c45b7ae17b7369ecca170" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 32935 } } peers { permanent_uuid: "d988027320354465b79936bb43b5a4ce" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 40851 } } peers { permanent_uuid: "af7611f4cfac4df6b1758c37d4ce6051" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45899 } }
I20260929 11:03:05.459260 13249 catalog_manager.cc:5495] T 656dad5e4bec41e48271c2aab461bd3b P d988027320354465b79936bb43b5a4ce reported cstate change: term changed from 0 to 1, leader changed from <none> to d988027320354465b79936bb43b5a4ce (127.26.120.194). New cstate: current_term: 1 leader_uuid: "d988027320354465b79936bb43b5a4ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc12e93b549c45b7ae17b7369ecca170" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 32935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d988027320354465b79936bb43b5a4ce" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 40851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af7611f4cfac4df6b1758c37d4ce6051" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45899 } health_report { overall_health: UNKNOWN } } }
I20260929 11:03:05.464836 13713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:05.462746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33966 (local address 127.26.120.194:40851)
0929 11:03:05.462784 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:05.462786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:05.462802 (+ 16us) server_negotiation.cc:407] Connection header received
0929 11:03:05.462850 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:05.462850 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:05.462866 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:05.462926 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 11:03:05.463029 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:05.463474 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:05.464197 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:05.464386 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:05.464440 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:05.464561 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 11:03:05.464642 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 11:03:05.464720 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:05.464754 (+ 34us) server_negotiation.cc:299] Negotiation successful
0929 11:03:05.464762 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 11:03:05.466563 13654 raft_consensus.cc:1261] T 656dad5e4bec41e48271c2aab461bd3b P af7611f4cfac4df6b1758c37d4ce6051 [term 1 FOLLOWER]: Refusing update from remote peer d988027320354465b79936bb43b5a4ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 11:03:05.466626 13392 raft_consensus.cc:1261] T 656dad5e4bec41e48271c2aab461bd3b P fc12e93b549c45b7ae17b7369ecca170 [term 1 FOLLOWER]: Refusing update from remote peer d988027320354465b79936bb43b5a4ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 11:03:05.466934 13724 consensus_queue.cc:1035] T 656dad5e4bec41e48271c2aab461bd3b P d988027320354465b79936bb43b5a4ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "af7611f4cfac4df6b1758c37d4ce6051" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 45899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 11:03:05.467047 13722 consensus_queue.cc:1035] T 656dad5e4bec41e48271c2aab461bd3b P d988027320354465b79936bb43b5a4ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc12e93b549c45b7ae17b7369ecca170" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 32935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 11:03:05.470036 13729 mvcc.cc:205] Tried to move back new op lower bound from 7334624401268322304 to 7334624401238179840. Current Snapshot: MvccSnapshot[applied={T|T < 7334624401268322304}]
I20260929 11:03:05.471436 13250 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 11:03:05.471529 13250 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 11:03:05.472251 13250 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1a646b041e6748bab77646f4355d1317: Sending DeleteTablet for 3 replicas of tablet 656dad5e4bec41e48271c2aab461bd3b
I20260929 11:03:05.472577 13372 tablet_service.cc:1514] Processing DeleteTablet for tablet 656dad5e4bec41e48271c2aab461bd3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:03:05 UTC) from {username='slave'} at 127.0.0.1:42536
I20260929 11:03:05.472599 13634 tablet_service.cc:1514] Processing DeleteTablet for tablet 656dad5e4bec41e48271c2aab461bd3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:03:05 UTC) from {username='slave'} at 127.0.0.1:52128
I20260929 11:03:05.472900 13738 tablet_replica.cc:330] T 656dad5e4bec41e48271c2aab461bd3b P af7611f4cfac4df6b1758c37d4ce6051: stopping tablet replica
I20260929 11:03:05.472924 13503 tablet_service.cc:1514] Processing DeleteTablet for tablet 656dad5e4bec41e48271c2aab461bd3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:03:05 UTC) from {username='slave'} at 127.0.0.1:33950
I20260929 11:03:05.472973 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 13310
I20260929 11:03:05.473001 13738 raft_consensus.cc:2229] T 656dad5e4bec41e48271c2aab461bd3b P af7611f4cfac4df6b1758c37d4ce6051 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 11:03:05.473099 13738 raft_consensus.cc:2258] T 656dad5e4bec41e48271c2aab461bd3b P af7611f4cfac4df6b1758c37d4ce6051 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 11:03:05.474296 13738 ts_tablet_manager.cc:1905] T 656dad5e4bec41e48271c2aab461bd3b P af7611f4cfac4df6b1758c37d4ce6051: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 11:03:05.476332 13738 ts_tablet_manager.cc:1918] T 656dad5e4bec41e48271c2aab461bd3b P af7611f4cfac4df6b1758c37d4ce6051: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 11:03:05.476399 13738 log.cc:1196] T 656dad5e4bec41e48271c2aab461bd3b P af7611f4cfac4df6b1758c37d4ce6051: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790679549986039-27107-0/minicluster-data/ts-2/wal/wals/656dad5e4bec41e48271c2aab461bd3b
I20260929 11:03:05.476660 13738 ts_tablet_manager.cc:1939] T 656dad5e4bec41e48271c2aab461bd3b P af7611f4cfac4df6b1758c37d4ce6051: Deleting consensus metadata
I20260929 11:03:05.477521 13235 catalog_manager.cc:4841] TS af7611f4cfac4df6b1758c37d4ce6051 (127.26.120.195:45899): tablet 656dad5e4bec41e48271c2aab461bd3b (table test-table [id=6a0c6f11bf484651a49aa8cc2e80bac0]) successfully deleted
W20260929 11:03:05.480583 13237 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.120.193:32935 (error 108)
I20260929 11:03:05.481153 13709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:05.481003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:32935 (local address 127.0.0.1:42550)
0929 11:03:05.481067 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:05.481116 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.193:32935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260929 11:03:05.481222 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 13441
I20260929 11:03:05.481244 13739 tablet_replica.cc:330] T 656dad5e4bec41e48271c2aab461bd3b P d988027320354465b79936bb43b5a4ce: stopping tablet replica
I20260929 11:03:05.481374 13739 raft_consensus.cc:2229] T 656dad5e4bec41e48271c2aab461bd3b P d988027320354465b79936bb43b5a4ce [term 1 LEADER]: Raft consensus shutting down.
I20260929 11:03:05.481487 13739 raft_consensus.cc:2258] T 656dad5e4bec41e48271c2aab461bd3b P d988027320354465b79936bb43b5a4ce [term 1 FOLLOWER]: Raft consensus is shut down!
W20260929 11:03:05.482671 13237 catalog_manager.cc:4584] TS fc12e93b549c45b7ae17b7369ecca170 (127.26.120.193:32935): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 656dad5e4bec41e48271c2aab461bd3b: Network error: Client connection negotiation failed: client connection to 127.26.120.193:32935: connect: Connection refused (error 111)
I20260929 11:03:05.486840 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 13572
I20260929 11:03:05.486924 13709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:05.486790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:40851 (local address 127.0.0.1:33980)
0929 11:03:05.486834 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:05.486868 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.194:40851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260929 11:03:05.491325 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 13219
2026-09-29T11:03:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (801 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 11:03:05 dist-test-slave-2lzq krb5kdc[13743](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 11:03:05 dist-test-slave-2lzq krb5kdc[13743](info): set up 2 sockets
Sep 29 11:03:05 dist-test-slave-2lzq krb5kdc[13743](info): commencing operation
krb5kdc: starting...
W20260929 11:03:06.530375 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 11:03:06 dist-test-slave-2lzq krb5kdc[13743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T11:03:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T11:03:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.120.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
I20260929 11:03:06.688555 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:36237
--webserver_interface=127.26.120.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:37313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:36237
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:03:06.772312 13756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:06.772532 13756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:06.772554 13756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:06.773733 13756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:03:06.773777 13756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:06.773792 13756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:06.773804 13756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:03:06.773856 13756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:03:06.775329 13756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:37313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:36237
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:36237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:06.775590 13756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:06.775823 13756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 11:03:06.780814 13756 server_base.cc:1034] running on GCE node
W20260929 11:03:06.780808 13761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:06.780799 13762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:06.780789 13764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:06.781293 13756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:06.781574 13756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:06.782724 13756 hybrid_clock.cc:648] HybridClock initialized: now 1790679786782708 us; error 34 us; skew 500 ppm
Sep 29 11:03:06 dist-test-slave-2lzq krb5kdc[13743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679786, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:03:06.785236 13756 init.cc:377] Logged in from keytab as kudu/127.26.120.254@KRBTEST.COM (short username kudu)
I20260929 11:03:06.785668 13756 webserver.cc:460] Webserver started at http://127.26.120.254:37485/ using document root <none> and password file <none>
I20260929 11:03:06.785904 13756 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:06.785972 13756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:06.786147 13756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:06.787220 13756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "30efad2fc862458f91bd4963563e4cd8"
format_stamp: "Formatted at 2026-09-29 11:03:06 on dist-test-slave-2lzq"
server_key: "47cc60c4c7cd3bc76b76aceeb17e0415"
server_key_iv: "4b77fd3a4466f7e7fec7a37b8f3f0700"
server_key_version: "encryptionkey@0"
I20260929 11:03:06.787585 13756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "30efad2fc862458f91bd4963563e4cd8"
format_stamp: "Formatted at 2026-09-29 11:03:06 on dist-test-slave-2lzq"
server_key: "47cc60c4c7cd3bc76b76aceeb17e0415"
server_key_iv: "4b77fd3a4466f7e7fec7a37b8f3f0700"
server_key_version: "encryptionkey@0"
I20260929 11:03:06.788913 13756 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 11:03:06.789623 13771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:06.789786 13756 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 11:03:06.789886 13756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "30efad2fc862458f91bd4963563e4cd8"
format_stamp: "Formatted at 2026-09-29 11:03:06 on dist-test-slave-2lzq"
server_key: "47cc60c4c7cd3bc76b76aceeb17e0415"
server_key_iv: "4b77fd3a4466f7e7fec7a37b8f3f0700"
server_key_version: "encryptionkey@0"
I20260929 11:03:06.789978 13756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:06.810748 13756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:06.812578 13756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:06.812750 13756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:06.816884 13756 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:36237
I20260929 11:03:06.816937 13823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:36237 every 8 connection(s)
I20260929 11:03:06.817333 13756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:03:06.818288 13824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:06.820468 13824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30efad2fc862458f91bd4963563e4cd8: Bootstrap starting.
I20260929 11:03:06.820992 13824 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30efad2fc862458f91bd4963563e4cd8: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:06.821218 13824 log.cc:824] T 00000000000000000000000000000000 P 30efad2fc862458f91bd4963563e4cd8: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:06.821873 13824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30efad2fc862458f91bd4963563e4cd8: No bootstrap required, opened a new log
I20260929 11:03:06.823529 13824 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30efad2fc862458f91bd4963563e4cd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30efad2fc862458f91bd4963563e4cd8" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 36237 } }
I20260929 11:03:06.823678 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 13756
I20260929 11:03:06.823729 13824 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30efad2fc862458f91bd4963563e4cd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:06.823781 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 11:03:06.823771 13824 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30efad2fc862458f91bd4963563e4cd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30efad2fc862458f91bd4963563e4cd8, State: Initialized, Role: FOLLOWER
I20260929 11:03:06.823933 13824 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30efad2fc862458f91bd4963563e4cd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30efad2fc862458f91bd4963563e4cd8" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 36237 } }
I20260929 11:03:06.823978 27107 external_mini_cluster.cc:1428] Setting key 6de64aeeede711ed415c86c49b542e3f
I20260929 11:03:06.824018 13824 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30efad2fc862458f91bd4963563e4cd8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 11:03:06.824052 13824 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30efad2fc862458f91bd4963563e4cd8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 11:03:06.824105 13824 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30efad2fc862458f91bd4963563e4cd8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:06.825212 13824 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30efad2fc862458f91bd4963563e4cd8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30efad2fc862458f91bd4963563e4cd8" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 36237 } }
I20260929 11:03:06.825325 13824 leader_election.cc:302] T 00000000000000000000000000000000 P 30efad2fc862458f91bd4963563e4cd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30efad2fc862458f91bd4963563e4cd8; no voters:
I20260929 11:03:06.825492 13824 leader_election.cc:288] T 00000000000000000000000000000000 P 30efad2fc862458f91bd4963563e4cd8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 11:03:06.825546 13829 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30efad2fc862458f91bd4963563e4cd8 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:03:06.825692 13829 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30efad2fc862458f91bd4963563e4cd8 [term 1 LEADER]: Becoming Leader. State: Replica: 30efad2fc862458f91bd4963563e4cd8, State: Running, Role: LEADER
I20260929 11:03:06.825743 13824 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30efad2fc862458f91bd4963563e4cd8 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:03:06.825836 13829 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30efad2fc862458f91bd4963563e4cd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30efad2fc862458f91bd4963563e4cd8" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 36237 } }
I20260929 11:03:06.826275 13831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30efad2fc862458f91bd4963563e4cd8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30efad2fc862458f91bd4963563e4cd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30efad2fc862458f91bd4963563e4cd8" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 36237 } } }
I20260929 11:03:06.826267 13829 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30efad2fc862458f91bd4963563e4cd8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30efad2fc862458f91bd4963563e4cd8. Latest consensus state: current_term: 1 leader_uuid: "30efad2fc862458f91bd4963563e4cd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30efad2fc862458f91bd4963563e4cd8" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 36237 } } }
I20260929 11:03:06.826362 13831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30efad2fc862458f91bd4963563e4cd8 [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:06.826371 13829 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30efad2fc862458f91bd4963563e4cd8 [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:06.827200 13838 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:03:06.827986 13838 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:03:06.829591 13838 catalog_manager.cc:1347] Generated new cluster ID: d8148b41bbb8465bbfeb69401095e61f
I20260929 11:03:06.829674 13838 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 11:03:06 dist-test-slave-2lzq krb5kdc[13743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:03:06.834600 13828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:06.824514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47218 (local address 127.26.120.254:36237)
0929 11:03:06.824728 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:06.824737 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:06.824770 (+ 33us) server_negotiation.cc:407] Connection header received
0929 11:03:06.824861 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:06.824866 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:06.824912 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:06.825019 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:06.825859 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:06.827415 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:06.828277 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:06.828719 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:06.828849 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:06.831351 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:06.831355 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:06.831357 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:06.831391 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:06.833041 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:06.833406 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:06.833411 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:06.833411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:06.833513 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:06.833728 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:06.833729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:06.833730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:06.833963 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:06.834154 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:06.834270 (+ 116us) server_negotiation.cc:299] Negotiation successful
0929 11:03:06.834422 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":136,"threads_started":1}
I20260929 11:03:06.854626 13838 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 11:03:06.855160 13838 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:03:06.866824 13838 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30efad2fc862458f91bd4963563e4cd8: Generated new TSK 0
I20260929 11:03:06.867033 13838 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.120.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab.
Entry for principal kudu/127.26.120.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab.
I20260929 11:03:06.896943 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.193:0
--local_ip_for_outbound_sockets=127.26.120.193
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:36237
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:37313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:03:06.993281 13850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:06.993474 13850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:06.993497 13850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:06.994720 13850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:06.994763 13850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:06.994789 13850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.193
I20260929 11:03:06.996217 13850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:37313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.193.keytab
--principal=kudu/127.26.120.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:36237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.193
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:06.996459 13850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:06.996698 13850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:07.001892 13858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:07.001892 13855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:07.002002 13850 server_base.cc:1034] running on GCE node
W20260929 11:03:07.001941 13856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:07.002497 13850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:07.002702 13850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:07.003868 13850 hybrid_clock.cc:648] HybridClock initialized: now 1790679787003849 us; error 34 us; skew 500 ppm
Sep 29 11:03:07 dist-test-slave-2lzq krb5kdc[13743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679787, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:03:07.006428 13850 init.cc:377] Logged in from keytab as kudu/127.26.120.193@KRBTEST.COM (short username kudu)
I20260929 11:03:07.006982 13850 webserver.cc:460] Webserver started at http://127.26.120.193:39071/ using document root <none> and password file <none>
I20260929 11:03:07.007230 13850 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:07.007277 13850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:07.007458 13850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:07.008550 13850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-0/data/instance:
uuid: "b0774f165f584b37b73205b4220eaa80"
format_stamp: "Formatted at 2026-09-29 11:03:07 on dist-test-slave-2lzq"
server_key: "024c4e5ff160ca6bfdf0f879592d1fe5"
server_key_iv: "a3da87ec55a43e1a479fadd401a6fb38"
server_key_version: "encryptionkey@0"
I20260929 11:03:07.008906 13850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance:
uuid: "b0774f165f584b37b73205b4220eaa80"
format_stamp: "Formatted at 2026-09-29 11:03:07 on dist-test-slave-2lzq"
server_key: "024c4e5ff160ca6bfdf0f879592d1fe5"
server_key_iv: "a3da87ec55a43e1a479fadd401a6fb38"
server_key_version: "encryptionkey@0"
I20260929 11:03:07.010244 13850 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 11:03:07.011924 13865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:07.012127 13850 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 11:03:07.012194 13850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-0/wal
uuid: "b0774f165f584b37b73205b4220eaa80"
format_stamp: "Formatted at 2026-09-29 11:03:07 on dist-test-slave-2lzq"
server_key: "024c4e5ff160ca6bfdf0f879592d1fe5"
server_key_iv: "a3da87ec55a43e1a479fadd401a6fb38"
server_key_version: "encryptionkey@0"
I20260929 11:03:07.012271 13850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:07.040553 13850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:07.042632 13850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:07.042807 13850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:07.043077 13850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:03:07.043399 13850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:03:07.043433 13850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:07.043457 13850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:03:07.043505 13850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:07.049199 13850 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.193:37611
I20260929 11:03:07.049250 13978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.193:37611 every 8 connection(s)
I20260929 11:03:07.049572 13850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
I20260929 11:03:07.052873 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 13850
I20260929 11:03:07.052989 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance
I20260929 11:03:07.053174 27107 external_mini_cluster.cc:1428] Setting key 28666475db4ae041d7dad253730735cf
Sep 29 11:03:07 dist-test-slave-2lzq krb5kdc[13743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679787, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.193@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:03:07.059618 13828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:07.050265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:40669 (local address 127.26.120.254:36237)
0929 11:03:07.050309 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:07.050311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:07.050528 (+ 217us) server_negotiation.cc:407] Connection header received
0929 11:03:07.051011 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:07.051012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:07.051026 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:07.051075 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:07.051955 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.052475 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.053892 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.054212 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.054282 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:07.056477 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:07.056478 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:07.056478 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:07.056485 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:07.057985 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:07.058536 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:07.058537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:07.058537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:07.058593 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:07.058928 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:07.058928 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:07.058929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:07.059039 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:07.059093 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:07.059465 (+ 372us) server_negotiation.cc:299] Negotiation successful
0929 11:03:07.059551 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 11:03:07.059949 13981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:07.050258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:36237 (local address 127.26.120.193:40669)
0929 11:03:07.050426 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:07.050462 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:07.050925 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:07.051126 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:07.051130 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:07.051678 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:03:07.051879 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:07.051884 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.052552 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.052553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:07.053790 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:07.053793 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.054317 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.054318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:07.054430 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:07.055001 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:03:07.055010 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:03:07.056390 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:03:07.058100 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:07.058109 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:07.058112 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:07.058439 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:07.058680 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:07.058681 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:07.058682 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:07.058842 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:07.059109 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:03:07.059117 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:03:07.059305 (+ 188us) client_negotiation.cc:769] Sending connection context
0929 11:03:07.059438 (+ 133us) client_negotiation.cc:240] Negotiation successful
0929 11:03:07.059640 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":88,"threads_started":1}
I20260929 11:03:07.060571 13979 heartbeater.cc:347] Connected to a master server at 127.26.120.254:36237
I20260929 11:03:07.060688 13979 heartbeater.cc:464] Registering TS with master...
I20260929 11:03:07.060902 13979 heartbeater.cc:511] Master 127.26.120.254:36237 requested a full tablet report, sending...
I20260929 11:03:07.061468 13788 ts_manager.cc:194] Registered new tserver with Master: b0774f165f584b37b73205b4220eaa80 (127.26.120.193:37611)
I20260929 11:03:07.062144 13788 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.193@KRBTEST.COM'} at 127.26.120.193:40669
WARNING: no policy specified for kudu/127.26.120.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab.
Entry for principal kudu/127.26.120.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab.
I20260929 11:03:07.079008 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.194:0
--local_ip_for_outbound_sockets=127.26.120.194
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:36237
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:37313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:03:07.160328 13984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:07.160521 13984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:07.160543 13984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:07.161662 13984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:07.161705 13984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:07.161754 13984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.194
I20260929 11:03:07.163189 13984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:37313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.194.keytab
--principal=kudu/127.26.120.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:36237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.194
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:07.163430 13984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:07.163647 13984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:07.168363 13989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:07.168478 13990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:07.168426 13992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:07.168546 13984 server_base.cc:1034] running on GCE node
I20260929 11:03:07.168730 13984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:07.168949 13984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:07.170113 13984 hybrid_clock.cc:648] HybridClock initialized: now 1790679787170098 us; error 28 us; skew 500 ppm
Sep 29 11:03:07 dist-test-slave-2lzq krb5kdc[13743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679787, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:03:07.172504 13984 init.cc:377] Logged in from keytab as kudu/127.26.120.194@KRBTEST.COM (short username kudu)
I20260929 11:03:07.172963 13984 webserver.cc:460] Webserver started at http://127.26.120.194:45679/ using document root <none> and password file <none>
I20260929 11:03:07.173182 13984 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:07.173249 13984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:07.173398 13984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:07.174456 13984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-1/data/instance:
uuid: "a89830c0890a40d5ae51d040e16b2faa"
format_stamp: "Formatted at 2026-09-29 11:03:07 on dist-test-slave-2lzq"
server_key: "faa6a29b43a11371dc995d151eb73c05"
server_key_iv: "f09cd2c1613160eafcf597940a46edd7"
server_key_version: "encryptionkey@0"
I20260929 11:03:07.174800 13984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance:
uuid: "a89830c0890a40d5ae51d040e16b2faa"
format_stamp: "Formatted at 2026-09-29 11:03:07 on dist-test-slave-2lzq"
server_key: "faa6a29b43a11371dc995d151eb73c05"
server_key_iv: "f09cd2c1613160eafcf597940a46edd7"
server_key_version: "encryptionkey@0"
I20260929 11:03:07.176065 13984 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 11:03:07.176745 13999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:07.176909 13984 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 11:03:07.177000 13984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-1/wal
uuid: "a89830c0890a40d5ae51d040e16b2faa"
format_stamp: "Formatted at 2026-09-29 11:03:07 on dist-test-slave-2lzq"
server_key: "faa6a29b43a11371dc995d151eb73c05"
server_key_iv: "f09cd2c1613160eafcf597940a46edd7"
server_key_version: "encryptionkey@0"
I20260929 11:03:07.177075 13984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:07.188220 13984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:07.190490 13984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:07.190644 13984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:07.190879 13984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:03:07.191180 13984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:03:07.191243 13984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:07.191288 13984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:03:07.191310 13984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:07.197213 13984 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.194:37371
I20260929 11:03:07.197257 14112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.194:37371 every 8 connection(s)
I20260929 11:03:07.197566 13984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
Sep 29 11:03:07 dist-test-slave-2lzq krb5kdc[13743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679787, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.194@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:03:07.203807 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 13984
I20260929 11:03:07.203897 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance
I20260929 11:03:07.204101 27107 external_mini_cluster.cc:1428] Setting key d08c88b1698b395bf6b3773f349d162f
I20260929 11:03:07.207345 13828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:07.198286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:52093 (local address 127.26.120.254:36237)
0929 11:03:07.198363 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:07.198365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:07.198580 (+ 215us) server_negotiation.cc:407] Connection header received
0929 11:03:07.199113 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:07.199114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:07.199124 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:07.199179 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:07.200051 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.200575 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.201644 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.201836 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.201890 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:07.204118 (+ 2228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:07.204120 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:07.204120 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:07.204127 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:07.205676 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:07.206153 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:07.206155 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:07.206155 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:07.206215 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:07.206550 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:07.206551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:07.206551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:07.206665 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:07.206731 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:07.207080 (+ 349us) server_negotiation.cc:299] Negotiation successful
0929 11:03:07.207176 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 11:03:07.207477 14115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:07.198275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:36237 (local address 127.26.120.194:52093)
0929 11:03:07.198456 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:07.198491 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:07.199025 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:07.199231 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:07.199234 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:07.199784 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:03:07.199975 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:07.199980 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.200692 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.200694 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:07.201565 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:07.201568 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.201911 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.201912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:07.202012 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:07.202550 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:03:07.202562 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:03:07.204021 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:03:07.205797 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:07.205804 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:07.205806 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:07.206044 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:07.206296 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:07.206297 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:07.206298 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:07.206471 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:07.206742 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:03:07.206748 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:03:07.206917 (+ 169us) client_negotiation.cc:769] Sending connection context
0929 11:03:07.207039 (+ 122us) client_negotiation.cc:240] Negotiation successful
0929 11:03:07.207214 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":101,"threads_started":1}
I20260929 11:03:07.207934 14113 heartbeater.cc:347] Connected to a master server at 127.26.120.254:36237
I20260929 11:03:07.208053 14113 heartbeater.cc:464] Registering TS with master...
I20260929 11:03:07.208235 14113 heartbeater.cc:511] Master 127.26.120.254:36237 requested a full tablet report, sending...
I20260929 11:03:07.208644 13788 ts_manager.cc:194] Registered new tserver with Master: a89830c0890a40d5ae51d040e16b2faa (127.26.120.194:37371)
I20260929 11:03:07.209198 13788 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.194@KRBTEST.COM'} at 127.26.120.194:52093
WARNING: no policy specified for kudu/127.26.120.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab.
Entry for principal kudu/127.26.120.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab.
I20260929 11:03:07.246500 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.195:0
--local_ip_for_outbound_sockets=127.26.120.195
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:36237
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:37313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:03:07.328218 14118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:07.328424 14118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:07.328449 14118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:07.329564 14118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:07.329602 14118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:07.329628 14118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.195
I20260929 11:03:07.331073 14118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:37313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.195.keytab
--principal=kudu/127.26.120.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:36237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.195
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:07.331276 14118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:07.331467 14118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:07.336410 14123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:07.336524 14118 server_base.cc:1034] running on GCE node
W20260929 11:03:07.336403 14124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:07.337626 14126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:07.337844 14118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:07.338147 14118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:07.339308 14118 hybrid_clock.cc:648] HybridClock initialized: now 1790679787339292 us; error 28 us; skew 500 ppm
Sep 29 11:03:07 dist-test-slave-2lzq krb5kdc[13743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679787, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:03:07.341718 14118 init.cc:377] Logged in from keytab as kudu/127.26.120.195@KRBTEST.COM (short username kudu)
I20260929 11:03:07.342262 14118 webserver.cc:460] Webserver started at http://127.26.120.195:38951/ using document root <none> and password file <none>
I20260929 11:03:07.342496 14118 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:07.342562 14118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:07.342718 14118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:07.343771 14118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-2/data/instance:
uuid: "4ac01fed6b3e49c8a3802a05021dd7cf"
format_stamp: "Formatted at 2026-09-29 11:03:07 on dist-test-slave-2lzq"
server_key: "14bb071913ce4d6fc8237c0683cbb0ed"
server_key_iv: "4fa34f9a9d263bbf5d221c38ad73291d"
server_key_version: "encryptionkey@0"
I20260929 11:03:07.344146 14118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance:
uuid: "4ac01fed6b3e49c8a3802a05021dd7cf"
format_stamp: "Formatted at 2026-09-29 11:03:07 on dist-test-slave-2lzq"
server_key: "14bb071913ce4d6fc8237c0683cbb0ed"
server_key_iv: "4fa34f9a9d263bbf5d221c38ad73291d"
server_key_version: "encryptionkey@0"
I20260929 11:03:07.345481 14118 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 11:03:07.346366 14133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:07.346531 14118 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:03:07.346630 14118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-2/wal
uuid: "4ac01fed6b3e49c8a3802a05021dd7cf"
format_stamp: "Formatted at 2026-09-29 11:03:07 on dist-test-slave-2lzq"
server_key: "14bb071913ce4d6fc8237c0683cbb0ed"
server_key_iv: "4fa34f9a9d263bbf5d221c38ad73291d"
server_key_version: "encryptionkey@0"
I20260929 11:03:07.346712 14118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:07.371816 14118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:07.373790 14118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:07.373958 14118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:07.374238 14118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:03:07.374563 14118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:03:07.374619 14118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:07.374653 14118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:03:07.374679 14118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:07.380563 14118 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.195:40641
I20260929 11:03:07.380605 14246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.195:40641 every 8 connection(s)
I20260929 11:03:07.381014 14118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
I20260929 11:03:07.381379 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 14118
I20260929 11:03:07.381470 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance
I20260929 11:03:07.381651 27107 external_mini_cluster.cc:1428] Setting key 3e912d3339e46745e209562ca9e19ac7
Sep 29 11:03:07 dist-test-slave-2lzq krb5kdc[13743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679787, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.195@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:03:07.389892 13828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:07.381723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:49501 (local address 127.26.120.254:36237)
0929 11:03:07.381793 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:07.381796 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:07.381957 (+ 161us) server_negotiation.cc:407] Connection header received
0929 11:03:07.382676 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:07.382677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:07.382686 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:07.382731 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:07.383629 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.384124 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.385068 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.385245 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.385294 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:07.387406 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:07.387407 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:07.387408 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:07.387414 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:07.388777 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:07.389128 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:07.389128 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:07.389129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:07.389179 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:07.389382 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:07.389383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:07.389383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:07.389481 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:07.389535 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:07.389758 (+ 223us) server_negotiation.cc:299] Negotiation successful
0929 11:03:07.389815 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 11:03:07.390057 14249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:07.381680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:36237 (local address 127.26.120.195:49501)
0929 11:03:07.381840 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:07.381883 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:07.382577 (+ 694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:07.382786 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:07.382790 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:07.383351 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:03:07.383554 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:07.383558 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.384201 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.384202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:07.384996 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:07.384998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.385315 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.385316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:07.385428 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:07.385873 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:03:07.385881 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:03:07.387320 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:03:07.388851 (+ 1531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:07.388865 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:07.388867 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:07.389067 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:07.389236 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:07.389237 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:07.389237 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:07.389329 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:07.389539 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:03:07.389543 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:03:07.389651 (+ 108us) client_negotiation.cc:769] Sending connection context
0929 11:03:07.389730 (+ 79us) client_negotiation.cc:240] Negotiation successful
0929 11:03:07.389845 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
I20260929 11:03:07.390601 14247 heartbeater.cc:347] Connected to a master server at 127.26.120.254:36237
I20260929 11:03:07.390710 14247 heartbeater.cc:464] Registering TS with master...
I20260929 11:03:07.390890 14247 heartbeater.cc:511] Master 127.26.120.254:36237 requested a full tablet report, sending...
I20260929 11:03:07.391266 13788 ts_manager.cc:194] Registered new tserver with Master: 4ac01fed6b3e49c8a3802a05021dd7cf (127.26.120.195:40641)
I20260929 11:03:07.391611 13788 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.120.195@KRBTEST.COM'} at 127.26.120.195:49501
I20260929 11:03:07.393330 27107 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 11:03:07.400202 13828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:07.394620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47222 (local address 127.26.120.254:36237)
0929 11:03:07.394660 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:07.394663 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:07.394789 (+ 126us) server_negotiation.cc:407] Connection header received
0929 11:03:07.394833 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:07.394833 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:07.394841 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:07.394886 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:07.395410 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.395876 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.396542 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.396714 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.396761 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:07.397836 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:07.397837 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:07.397838 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:07.397843 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:07.399258 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:07.399575 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:07.399575 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:07.399576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:07.399624 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:07.399771 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:07.399771 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:07.399772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:07.399904 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:07.399975 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:07.400061 (+ 86us) server_negotiation.cc:299] Negotiation successful
0929 11:03:07.400117 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 11:03:07.401738 13788 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47222:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 11:03:07.403417 13788 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 11:03:07.410002 14258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:07.405767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:37611 (local address 127.0.0.1:39966)
0929 11:03:07.406118 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:07.406162 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:07.406230 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:07.406838 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:07.406839 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:07.406842 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:03:07.406898 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:07.406900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.407893 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.407894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:07.408916 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:07.408919 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.409696 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.409697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:07.409776 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:07.409791 (+ 15us) client_negotiation.cc:769] Sending connection context
0929 11:03:07.409867 (+ 76us) client_negotiation.cc:240] Negotiation successful
0929 11:03:07.409923 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":89,"threads_started":1}
I20260929 11:03:07.410027 14260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:07.405894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39966 (local address 127.26.120.193:37611)
0929 11:03:07.406667 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:07.406672 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:07.406682 (+ 10us) server_negotiation.cc:407] Connection header received
0929 11:03:07.406719 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:07.406721 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:07.406736 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:07.406825 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:03:07.406976 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.407786 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.409012 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.409610 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.409686 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:07.409739 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:07.409900 (+ 161us) server_negotiation.cc:299] Negotiation successful
0929 11:03:07.409943 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":727,"thread_start_us":93,"threads_started":1}
I20260929 11:03:07.410534 14262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:07.405974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:37371 (local address 127.0.0.1:43662)
0929 11:03:07.406295 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:07.406305 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:07.406356 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:07.406578 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:07.406581 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:07.406583 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:03:07.406663 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:07.406665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.408046 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.408047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:07.409279 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:07.409282 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.410274 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.410275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:07.410339 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:07.410345 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 11:03:07.410408 (+ 63us) client_negotiation.cc:240] Negotiation successful
0929 11:03:07.410454 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":66,"thread_start_us":66,"threads_started":1}
I20260929 11:03:07.410782 14263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:07.406117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43662 (local address 127.26.120.194:37371)
0929 11:03:07.406306 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:07.406309 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:07.406347 (+ 38us) server_negotiation.cc:407] Connection header received
0929 11:03:07.406431 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:07.406434 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:07.406447 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:07.406523 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:03:07.406727 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.407941 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.409372 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.410099 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.410176 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:07.410236 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:07.410656 (+ 420us) server_negotiation.cc:299] Negotiation successful
0929 11:03:07.410703 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":90,"threads_started":1}
I20260929 11:03:07.410940 13913 tablet_service.cc:1467] Processing CreateTablet for tablet 85b07deaec4d4ae489e7f3f4bba34b34 (DEFAULT_TABLE table=test-table [id=02fbec8533564f2fbcdd38bedfade497]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:03:07.411280 13913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85b07deaec4d4ae489e7f3f4bba34b34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:07.413782 14264 tablet_bootstrap.cc:492] T 85b07deaec4d4ae489e7f3f4bba34b34 P b0774f165f584b37b73205b4220eaa80: Bootstrap starting.
I20260929 11:03:07.414197 14261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:07.405957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44448 (local address 127.26.120.195:40641)
0929 11:03:07.406161 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:07.406170 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:07.406374 (+ 204us) server_negotiation.cc:407] Connection header received
0929 11:03:07.406419 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:07.406422 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:07.406440 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:07.406522 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:03:07.406871 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.407920 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.409394 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.410025 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.410130 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:07.410193 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:07.410391 (+ 198us) server_negotiation.cc:299] Negotiation successful
0929 11:03:07.410440 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":71,"threads_started":1}
I20260929 11:03:07.414280 14047 tablet_service.cc:1467] Processing CreateTablet for tablet 85b07deaec4d4ae489e7f3f4bba34b34 (DEFAULT_TABLE table=test-table [id=02fbec8533564f2fbcdd38bedfade497]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:03:07.414433 14259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:07.405767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:40641 (local address 127.0.0.1:44448)
0929 11:03:07.406232 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:07.406245 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:07.406312 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:07.406744 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:07.406745 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:07.406746 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:03:07.406785 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:07.406786 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.408045 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.408046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:07.409279 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:07.409282 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.410227 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.410228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:07.410302 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:07.410305 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 11:03:07.414302 (+ 3997us) client_negotiation.cc:240] Negotiation successful
0929 11:03:07.414365 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"mutex_wait_us":33,"thread_start_us":68,"threads_started":1}
I20260929 11:03:07.414469 14264 tablet_bootstrap.cc:654] T 85b07deaec4d4ae489e7f3f4bba34b34 P b0774f165f584b37b73205b4220eaa80: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:07.414635 14047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85b07deaec4d4ae489e7f3f4bba34b34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:07.415004 14264 log.cc:824] T 85b07deaec4d4ae489e7f3f4bba34b34 P b0774f165f584b37b73205b4220eaa80: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:07.415772 14264 tablet_bootstrap.cc:492] T 85b07deaec4d4ae489e7f3f4bba34b34 P b0774f165f584b37b73205b4220eaa80: No bootstrap required, opened a new log
I20260929 11:03:07.415848 14264 ts_tablet_manager.cc:1397] T 85b07deaec4d4ae489e7f3f4bba34b34 P b0774f165f584b37b73205b4220eaa80: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 11:03:07.417879 14266 tablet_bootstrap.cc:492] T 85b07deaec4d4ae489e7f3f4bba34b34 P a89830c0890a40d5ae51d040e16b2faa: Bootstrap starting.
I20260929 11:03:07.418030 14264 raft_consensus.cc:348] T 85b07deaec4d4ae489e7f3f4bba34b34 P b0774f165f584b37b73205b4220eaa80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0774f165f584b37b73205b4220eaa80" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37611 } } peers { permanent_uuid: "4ac01fed6b3e49c8a3802a05021dd7cf" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 40641 } } peers { permanent_uuid: "a89830c0890a40d5ae51d040e16b2faa" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37371 } }
I20260929 11:03:07.418036 14181 tablet_service.cc:1467] Processing CreateTablet for tablet 85b07deaec4d4ae489e7f3f4bba34b34 (DEFAULT_TABLE table=test-table [id=02fbec8533564f2fbcdd38bedfade497]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:03:07.418244 14264 raft_consensus.cc:374] T 85b07deaec4d4ae489e7f3f4bba34b34 P b0774f165f584b37b73205b4220eaa80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:07.418286 14264 raft_consensus.cc:729] T 85b07deaec4d4ae489e7f3f4bba34b34 P b0774f165f584b37b73205b4220eaa80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0774f165f584b37b73205b4220eaa80, State: Initialized, Role: FOLLOWER
I20260929 11:03:07.418411 14264 consensus_queue.cc:260] T 85b07deaec4d4ae489e7f3f4bba34b34 P b0774f165f584b37b73205b4220eaa80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0774f165f584b37b73205b4220eaa80" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37611 } } peers { permanent_uuid: "4ac01fed6b3e49c8a3802a05021dd7cf" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 40641 } } peers { permanent_uuid: "a89830c0890a40d5ae51d040e16b2faa" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37371 } }
I20260929 11:03:07.418467 14181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85b07deaec4d4ae489e7f3f4bba34b34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:07.418588 14266 tablet_bootstrap.cc:654] T 85b07deaec4d4ae489e7f3f4bba34b34 P a89830c0890a40d5ae51d040e16b2faa: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:07.418704 14264 ts_tablet_manager.cc:1428] T 85b07deaec4d4ae489e7f3f4bba34b34 P b0774f165f584b37b73205b4220eaa80: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260929 11:03:07.418723 13979 heartbeater.cc:503] Master 127.26.120.254:36237 was elected leader, sending a full tablet report...
I20260929 11:03:07.419281 14266 log.cc:824] T 85b07deaec4d4ae489e7f3f4bba34b34 P a89830c0890a40d5ae51d040e16b2faa: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:07.420092 14266 tablet_bootstrap.cc:492] T 85b07deaec4d4ae489e7f3f4bba34b34 P a89830c0890a40d5ae51d040e16b2faa: No bootstrap required, opened a new log
I20260929 11:03:07.420179 14266 ts_tablet_manager.cc:1397] T 85b07deaec4d4ae489e7f3f4bba34b34 P a89830c0890a40d5ae51d040e16b2faa: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 11:03:07.421468 14269 tablet_bootstrap.cc:492] T 85b07deaec4d4ae489e7f3f4bba34b34 P 4ac01fed6b3e49c8a3802a05021dd7cf: Bootstrap starting.
I20260929 11:03:07.421597 14266 raft_consensus.cc:348] T 85b07deaec4d4ae489e7f3f4bba34b34 P a89830c0890a40d5ae51d040e16b2faa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0774f165f584b37b73205b4220eaa80" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37611 } } peers { permanent_uuid: "4ac01fed6b3e49c8a3802a05021dd7cf" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 40641 } } peers { permanent_uuid: "a89830c0890a40d5ae51d040e16b2faa" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37371 } }
I20260929 11:03:07.421775 14266 raft_consensus.cc:374] T 85b07deaec4d4ae489e7f3f4bba34b34 P a89830c0890a40d5ae51d040e16b2faa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:07.421821 14266 raft_consensus.cc:729] T 85b07deaec4d4ae489e7f3f4bba34b34 P a89830c0890a40d5ae51d040e16b2faa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a89830c0890a40d5ae51d040e16b2faa, State: Initialized, Role: FOLLOWER
I20260929 11:03:07.421939 14266 consensus_queue.cc:260] T 85b07deaec4d4ae489e7f3f4bba34b34 P a89830c0890a40d5ae51d040e16b2faa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0774f165f584b37b73205b4220eaa80" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37611 } } peers { permanent_uuid: "4ac01fed6b3e49c8a3802a05021dd7cf" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 40641 } } peers { permanent_uuid: "a89830c0890a40d5ae51d040e16b2faa" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37371 } }
I20260929 11:03:07.422175 14269 tablet_bootstrap.cc:654] T 85b07deaec4d4ae489e7f3f4bba34b34 P 4ac01fed6b3e49c8a3802a05021dd7cf: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:07.422236 14266 ts_tablet_manager.cc:1428] T 85b07deaec4d4ae489e7f3f4bba34b34 P a89830c0890a40d5ae51d040e16b2faa: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260929 11:03:07.422282 14113 heartbeater.cc:503] Master 127.26.120.254:36237 was elected leader, sending a full tablet report...
I20260929 11:03:07.422667 14269 log.cc:824] T 85b07deaec4d4ae489e7f3f4bba34b34 P 4ac01fed6b3e49c8a3802a05021dd7cf: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:07.423346 14269 tablet_bootstrap.cc:492] T 85b07deaec4d4ae489e7f3f4bba34b34 P 4ac01fed6b3e49c8a3802a05021dd7cf: No bootstrap required, opened a new log
I20260929 11:03:07.423425 14269 ts_tablet_manager.cc:1397] T 85b07deaec4d4ae489e7f3f4bba34b34 P 4ac01fed6b3e49c8a3802a05021dd7cf: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260929 11:03:07.424665 14269 raft_consensus.cc:348] T 85b07deaec4d4ae489e7f3f4bba34b34 P 4ac01fed6b3e49c8a3802a05021dd7cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0774f165f584b37b73205b4220eaa80" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37611 } } peers { permanent_uuid: "4ac01fed6b3e49c8a3802a05021dd7cf" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 40641 } } peers { permanent_uuid: "a89830c0890a40d5ae51d040e16b2faa" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37371 } }
I20260929 11:03:07.424813 14269 raft_consensus.cc:374] T 85b07deaec4d4ae489e7f3f4bba34b34 P 4ac01fed6b3e49c8a3802a05021dd7cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:07.424880 14269 raft_consensus.cc:729] T 85b07deaec4d4ae489e7f3f4bba34b34 P 4ac01fed6b3e49c8a3802a05021dd7cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ac01fed6b3e49c8a3802a05021dd7cf, State: Initialized, Role: FOLLOWER
I20260929 11:03:07.424993 14269 consensus_queue.cc:260] T 85b07deaec4d4ae489e7f3f4bba34b34 P 4ac01fed6b3e49c8a3802a05021dd7cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0774f165f584b37b73205b4220eaa80" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37611 } } peers { permanent_uuid: "4ac01fed6b3e49c8a3802a05021dd7cf" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 40641 } } peers { permanent_uuid: "a89830c0890a40d5ae51d040e16b2faa" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37371 } }
I20260929 11:03:07.425266 14269 ts_tablet_manager.cc:1428] T 85b07deaec4d4ae489e7f3f4bba34b34 P 4ac01fed6b3e49c8a3802a05021dd7cf: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260929 11:03:07.425303 14247 heartbeater.cc:503] Master 127.26.120.254:36237 was elected leader, sending a full tablet report...
W20260929 11:03:07.448112 14114 tablet.cc:2369] T 85b07deaec4d4ae489e7f3f4bba34b34 P a89830c0890a40d5ae51d040e16b2faa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 11:03:07.534829 14272 raft_consensus.cc:482] T 85b07deaec4d4ae489e7f3f4bba34b34 P 4ac01fed6b3e49c8a3802a05021dd7cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 11:03:07.534988 14272 raft_consensus.cc:504] T 85b07deaec4d4ae489e7f3f4bba34b34 P 4ac01fed6b3e49c8a3802a05021dd7cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0774f165f584b37b73205b4220eaa80" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37611 } } peers { permanent_uuid: "4ac01fed6b3e49c8a3802a05021dd7cf" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 40641 } } peers { permanent_uuid: "a89830c0890a40d5ae51d040e16b2faa" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37371 } }
I20260929 11:03:07.535382 14272 leader_election.cc:288] T 85b07deaec4d4ae489e7f3f4bba34b34 P 4ac01fed6b3e49c8a3802a05021dd7cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0774f165f584b37b73205b4220eaa80 (127.26.120.193:37611), a89830c0890a40d5ae51d040e16b2faa (127.26.120.194:37371)
I20260929 11:03:07.538612 14249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:07.535639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:37611 (local address 127.26.120.195:50069)
0929 11:03:07.535702 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:07.535720 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:07.535803 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:07.536041 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:07.536043 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:07.536045 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:03:07.536126 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:07.536128 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.536853 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.536854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:07.537646 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:07.537649 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.538333 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.538334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:07.538417 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:07.538421 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 11:03:07.538486 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 11:03:07.538536 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260929 11:03:07.538637 14260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:07.535727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:50069 (local address 127.26.120.193:37611)
0929 11:03:07.535802 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:07.535804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:07.535815 (+ 11us) server_negotiation.cc:407] Connection header received
0929 11:03:07.535896 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:07.535897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:07.535906 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:07.535994 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:03:07.536205 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.536755 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.537778 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.538254 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.538312 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:07.538320 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:07.538497 (+ 177us) server_negotiation.cc:299] Negotiation successful
0929 11:03:07.538560 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 11:03:07.538890 14273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:07.535798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:37371 (local address 127.26.120.195:58357)
0929 11:03:07.535969 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:07.535978 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:07.536030 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:07.536176 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:07.536177 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:07.536179 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:03:07.536251 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:07.536253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.536977 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.536977 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:07.537842 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:07.537844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.538561 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.538562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:07.538652 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:07.538657 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 11:03:07.538772 (+ 115us) client_negotiation.cc:240] Negotiation successful
0929 11:03:07.538814 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":14,"thread_start_us":117,"threads_started":1}
I20260929 11:03:07.538887 14263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:07.535859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:58357 (local address 127.26.120.194:37371)
0929 11:03:07.535919 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:07.535921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:07.536037 (+ 116us) server_negotiation.cc:407] Connection header received
0929 11:03:07.536074 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:07.536075 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:07.536084 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:07.536153 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:03:07.536368 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.536871 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.537931 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.538478 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.538541 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:07.538548 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:07.538789 (+ 241us) server_negotiation.cc:299] Negotiation successful
0929 11:03:07.538827 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 11:03:07.539050 13933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85b07deaec4d4ae489e7f3f4bba34b34" candidate_uuid: "4ac01fed6b3e49c8a3802a05021dd7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0774f165f584b37b73205b4220eaa80" is_pre_election: true
I20260929 11:03:07.539239 13933 raft_consensus.cc:2454] T 85b07deaec4d4ae489e7f3f4bba34b34 P b0774f165f584b37b73205b4220eaa80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ac01fed6b3e49c8a3802a05021dd7cf in term 0.
I20260929 11:03:07.539372 14067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85b07deaec4d4ae489e7f3f4bba34b34" candidate_uuid: "4ac01fed6b3e49c8a3802a05021dd7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a89830c0890a40d5ae51d040e16b2faa" is_pre_election: true
I20260929 11:03:07.539479 14134 leader_election.cc:302] T 85b07deaec4d4ae489e7f3f4bba34b34 P 4ac01fed6b3e49c8a3802a05021dd7cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ac01fed6b3e49c8a3802a05021dd7cf, b0774f165f584b37b73205b4220eaa80; no voters:
I20260929 11:03:07.539542 14067 raft_consensus.cc:2454] T 85b07deaec4d4ae489e7f3f4bba34b34 P a89830c0890a40d5ae51d040e16b2faa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ac01fed6b3e49c8a3802a05021dd7cf in term 0.
I20260929 11:03:07.539649 14272 raft_consensus.cc:2781] T 85b07deaec4d4ae489e7f3f4bba34b34 P 4ac01fed6b3e49c8a3802a05021dd7cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 11:03:07.539721 14272 raft_consensus.cc:482] T 85b07deaec4d4ae489e7f3f4bba34b34 P 4ac01fed6b3e49c8a3802a05021dd7cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 11:03:07.539755 14272 raft_consensus.cc:3037] T 85b07deaec4d4ae489e7f3f4bba34b34 P 4ac01fed6b3e49c8a3802a05021dd7cf [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:07.540452 14272 raft_consensus.cc:504] T 85b07deaec4d4ae489e7f3f4bba34b34 P 4ac01fed6b3e49c8a3802a05021dd7cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0774f165f584b37b73205b4220eaa80" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37611 } } peers { permanent_uuid: "4ac01fed6b3e49c8a3802a05021dd7cf" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 40641 } } peers { permanent_uuid: "a89830c0890a40d5ae51d040e16b2faa" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37371 } }
I20260929 11:03:07.540583 14272 leader_election.cc:288] T 85b07deaec4d4ae489e7f3f4bba34b34 P 4ac01fed6b3e49c8a3802a05021dd7cf [CANDIDATE]: Term 1 election: Requested vote from peers b0774f165f584b37b73205b4220eaa80 (127.26.120.193:37611), a89830c0890a40d5ae51d040e16b2faa (127.26.120.194:37371)
I20260929 11:03:07.540771 14067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85b07deaec4d4ae489e7f3f4bba34b34" candidate_uuid: "4ac01fed6b3e49c8a3802a05021dd7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a89830c0890a40d5ae51d040e16b2faa"
I20260929 11:03:07.540758 13933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85b07deaec4d4ae489e7f3f4bba34b34" candidate_uuid: "4ac01fed6b3e49c8a3802a05021dd7cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0774f165f584b37b73205b4220eaa80"
I20260929 11:03:07.540846 13933 raft_consensus.cc:3037] T 85b07deaec4d4ae489e7f3f4bba34b34 P b0774f165f584b37b73205b4220eaa80 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:07.540846 14067 raft_consensus.cc:3037] T 85b07deaec4d4ae489e7f3f4bba34b34 P a89830c0890a40d5ae51d040e16b2faa [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:07.541762 13933 raft_consensus.cc:2454] T 85b07deaec4d4ae489e7f3f4bba34b34 P b0774f165f584b37b73205b4220eaa80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ac01fed6b3e49c8a3802a05021dd7cf in term 1.
I20260929 11:03:07.541762 14067 raft_consensus.cc:2454] T 85b07deaec4d4ae489e7f3f4bba34b34 P a89830c0890a40d5ae51d040e16b2faa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ac01fed6b3e49c8a3802a05021dd7cf in term 1.
I20260929 11:03:07.541946 14134 leader_election.cc:302] T 85b07deaec4d4ae489e7f3f4bba34b34 P 4ac01fed6b3e49c8a3802a05021dd7cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ac01fed6b3e49c8a3802a05021dd7cf, b0774f165f584b37b73205b4220eaa80; no voters:
I20260929 11:03:07.542075 14272 raft_consensus.cc:2781] T 85b07deaec4d4ae489e7f3f4bba34b34 P 4ac01fed6b3e49c8a3802a05021dd7cf [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:03:07.542243 14272 raft_consensus.cc:686] T 85b07deaec4d4ae489e7f3f4bba34b34 P 4ac01fed6b3e49c8a3802a05021dd7cf [term 1 LEADER]: Becoming Leader. State: Replica: 4ac01fed6b3e49c8a3802a05021dd7cf, State: Running, Role: LEADER
I20260929 11:03:07.542342 14272 consensus_queue.cc:237] T 85b07deaec4d4ae489e7f3f4bba34b34 P 4ac01fed6b3e49c8a3802a05021dd7cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0774f165f584b37b73205b4220eaa80" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37611 } } peers { permanent_uuid: "4ac01fed6b3e49c8a3802a05021dd7cf" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 40641 } } peers { permanent_uuid: "a89830c0890a40d5ae51d040e16b2faa" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37371 } }
I20260929 11:03:07.543208 13788 catalog_manager.cc:5495] T 85b07deaec4d4ae489e7f3f4bba34b34 P 4ac01fed6b3e49c8a3802a05021dd7cf reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ac01fed6b3e49c8a3802a05021dd7cf (127.26.120.195). New cstate: current_term: 1 leader_uuid: "4ac01fed6b3e49c8a3802a05021dd7cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0774f165f584b37b73205b4220eaa80" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ac01fed6b3e49c8a3802a05021dd7cf" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 40641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a89830c0890a40d5ae51d040e16b2faa" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37371 } health_report { overall_health: UNKNOWN } } }
W20260929 11:03:07.550256 13980 tablet.cc:2369] T 85b07deaec4d4ae489e7f3f4bba34b34 P b0774f165f584b37b73205b4220eaa80: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 11:03:07.554433 14261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:07.552018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44454 (local address 127.26.120.195:40641)
0929 11:03:07.552061 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:07.552063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:07.552070 (+ 7us) server_negotiation.cc:407] Connection header received
0929 11:03:07.552145 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:07.552145 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:07.552162 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:07.552219 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 11:03:07.552321 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.552778 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.553519 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.553707 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.553765 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:07.553952 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 11:03:07.554101 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 11:03:07.554169 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:07.554317 (+ 148us) server_negotiation.cc:299] Negotiation successful
0929 11:03:07.554336 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 11:03:07.556299 14067 raft_consensus.cc:1261] T 85b07deaec4d4ae489e7f3f4bba34b34 P a89830c0890a40d5ae51d040e16b2faa [term 1 FOLLOWER]: Refusing update from remote peer 4ac01fed6b3e49c8a3802a05021dd7cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 11:03:07.556325 13933 raft_consensus.cc:1261] T 85b07deaec4d4ae489e7f3f4bba34b34 P b0774f165f584b37b73205b4220eaa80 [term 1 FOLLOWER]: Refusing update from remote peer 4ac01fed6b3e49c8a3802a05021dd7cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 11:03:07.556677 14272 consensus_queue.cc:1035] T 85b07deaec4d4ae489e7f3f4bba34b34 P 4ac01fed6b3e49c8a3802a05021dd7cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0774f165f584b37b73205b4220eaa80" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 37611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 11:03:07.556833 14274 consensus_queue.cc:1035] T 85b07deaec4d4ae489e7f3f4bba34b34 P 4ac01fed6b3e49c8a3802a05021dd7cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a89830c0890a40d5ae51d040e16b2faa" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 37371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 11:03:07.559631 14279 mvcc.cc:205] Tried to move back new op lower bound from 7334624409827811328 to 7334624409774092288. Current Snapshot: MvccSnapshot[applied={T|T < 7334624409827811328}]
I20260929 11:03:07.561959 13788 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47222:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 11:03:07.562089 13788 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47222:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 11:03:07.562906 13788 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 30efad2fc862458f91bd4963563e4cd8: Sending DeleteTablet for 3 replicas of tablet 85b07deaec4d4ae489e7f3f4bba34b34
I20260929 11:03:07.563244 14181 tablet_service.cc:1514] Processing DeleteTablet for tablet 85b07deaec4d4ae489e7f3f4bba34b34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:03:07 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:44448
I20260929 11:03:07.563301 13913 tablet_service.cc:1514] Processing DeleteTablet for tablet 85b07deaec4d4ae489e7f3f4bba34b34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:03:07 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:39966
I20260929 11:03:07.563303 14047 tablet_service.cc:1514] Processing DeleteTablet for tablet 85b07deaec4d4ae489e7f3f4bba34b34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:03:07 UTC) from {username='kudu', principal='kudu/127.26.120.254@KRBTEST.COM'} at 127.0.0.1:43662
I20260929 11:03:07.563544 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 13850
I20260929 11:03:07.564392 14289 tablet_replica.cc:330] T 85b07deaec4d4ae489e7f3f4bba34b34 P a89830c0890a40d5ae51d040e16b2faa: stopping tablet replica
I20260929 11:03:07.564515 14289 raft_consensus.cc:2229] T 85b07deaec4d4ae489e7f3f4bba34b34 P a89830c0890a40d5ae51d040e16b2faa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 11:03:07.564632 14289 raft_consensus.cc:2258] T 85b07deaec4d4ae489e7f3f4bba34b34 P a89830c0890a40d5ae51d040e16b2faa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 11:03:07.564726 14287 tablet_replica.cc:330] T 85b07deaec4d4ae489e7f3f4bba34b34 P 4ac01fed6b3e49c8a3802a05021dd7cf: stopping tablet replica
I20260929 11:03:07.564852 14287 raft_consensus.cc:2229] T 85b07deaec4d4ae489e7f3f4bba34b34 P 4ac01fed6b3e49c8a3802a05021dd7cf [term 1 LEADER]: Raft consensus shutting down.
I20260929 11:03:07.565003 14287 raft_consensus.cc:2258] T 85b07deaec4d4ae489e7f3f4bba34b34 P 4ac01fed6b3e49c8a3802a05021dd7cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 11:03:07.565554 14289 ts_tablet_manager.cc:1905] T 85b07deaec4d4ae489e7f3f4bba34b34 P a89830c0890a40d5ae51d040e16b2faa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 11:03:07.566322 14287 ts_tablet_manager.cc:1905] T 85b07deaec4d4ae489e7f3f4bba34b34 P 4ac01fed6b3e49c8a3802a05021dd7cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 11:03:07.567492 14289 ts_tablet_manager.cc:1918] T 85b07deaec4d4ae489e7f3f4bba34b34 P a89830c0890a40d5ae51d040e16b2faa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 11:03:07.567622 14289 log.cc:1196] T 85b07deaec4d4ae489e7f3f4bba34b34 P a89830c0890a40d5ae51d040e16b2faa: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-1/wal/wals/85b07deaec4d4ae489e7f3f4bba34b34
I20260929 11:03:07.567961 14289 ts_tablet_manager.cc:1939] T 85b07deaec4d4ae489e7f3f4bba34b34 P a89830c0890a40d5ae51d040e16b2faa: Deleting consensus metadata
I20260929 11:03:07.568070 14287 ts_tablet_manager.cc:1918] T 85b07deaec4d4ae489e7f3f4bba34b34 P 4ac01fed6b3e49c8a3802a05021dd7cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 11:03:07.568143 14287 log.cc:1196] T 85b07deaec4d4ae489e7f3f4bba34b34 P 4ac01fed6b3e49c8a3802a05021dd7cf: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790679549986039-27107-0/minicluster-data/ts-2/wal/wals/85b07deaec4d4ae489e7f3f4bba34b34
I20260929 11:03:07.568404 14287 ts_tablet_manager.cc:1939] T 85b07deaec4d4ae489e7f3f4bba34b34 P 4ac01fed6b3e49c8a3802a05021dd7cf: Deleting consensus metadata
I20260929 11:03:07.568492 13772 catalog_manager.cc:4841] TS a89830c0890a40d5ae51d040e16b2faa (127.26.120.194:37371): tablet 85b07deaec4d4ae489e7f3f4bba34b34 (table test-table [id=02fbec8533564f2fbcdd38bedfade497]) successfully deleted
I20260929 11:03:07.568819 13774 catalog_manager.cc:4841] TS 4ac01fed6b3e49c8a3802a05021dd7cf (127.26.120.195:40641): tablet 85b07deaec4d4ae489e7f3f4bba34b34 (table test-table [id=02fbec8533564f2fbcdd38bedfade497]) successfully deleted
W20260929 11:03:07.569968 13772 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.120.193:37611 (error 108)
I20260929 11:03:07.570551 14259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:07.570408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:37611 (local address 127.0.0.1:39974)
0929 11:03:07.570470 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:07.570512 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.193:37611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260929 11:03:07.570580 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 13984
W20260929 11:03:07.573410 13772 catalog_manager.cc:4584] TS b0774f165f584b37b73205b4220eaa80 (127.26.120.193:37611): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 85b07deaec4d4ae489e7f3f4bba34b34: Network error: Client connection negotiation failed: client connection to 127.26.120.193:37611: connect: Connection refused (error 111)
I20260929 11:03:07.577756 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 14118
I20260929 11:03:07.581611 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 13756
2026-09-29T11:03:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2091 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-29T11:03:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T11:03:07Z Disabled control of system clock
I20260929 11:03:07.609643 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:39781
--webserver_interface=127.26.120.254
--webserver_port=0
--builtin_ntp_servers=127.26.120.212:41397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:39781
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 11:03:07.689656 14294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:07.689878 14294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:07.689899 14294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:07.691090 14294 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:03:07.691134 14294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:07.691147 14294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:07.691196 14294 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:03:07.691221 14294 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:03:07.692682 14294 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:41397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:39781
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:39781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:07.692941 14294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:07.693253 14294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:07.698094 14300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:07.698084 14299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:07.698079 14302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:07.698305 14294 server_base.cc:1034] running on GCE node
I20260929 11:03:07.698529 14294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:07.698776 14294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:07.699955 14294 hybrid_clock.cc:648] HybridClock initialized: now 1790679787699930 us; error 39 us; skew 500 ppm
I20260929 11:03:07.701190 14294 webserver.cc:460] Webserver started at http://127.26.120.254:35033/ using document root <none> and password file <none>
I20260929 11:03:07.701479 14294 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:07.701550 14294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:07.701706 14294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:07.702884 14294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "fb803e86a6df419d956b9c4bc2879160"
format_stamp: "Formatted at 2026-09-29 11:03:07 on dist-test-slave-2lzq"
server_key: "03ed0b3d274ebe27eed610d4c08ff982"
server_key_iv: "74430fe032232dc02e906513e71d9fb4"
server_key_version: "encryptionkey@0"
I20260929 11:03:07.703240 14294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "fb803e86a6df419d956b9c4bc2879160"
format_stamp: "Formatted at 2026-09-29 11:03:07 on dist-test-slave-2lzq"
server_key: "03ed0b3d274ebe27eed610d4c08ff982"
server_key_iv: "74430fe032232dc02e906513e71d9fb4"
server_key_version: "encryptionkey@0"
I20260929 11:03:07.704515 14294 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 11:03:07.705250 14308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:07.705446 14294 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 11:03:07.705549 14294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "fb803e86a6df419d956b9c4bc2879160"
format_stamp: "Formatted at 2026-09-29 11:03:07 on dist-test-slave-2lzq"
server_key: "03ed0b3d274ebe27eed610d4c08ff982"
server_key_iv: "74430fe032232dc02e906513e71d9fb4"
server_key_version: "encryptionkey@0"
I20260929 11:03:07.705621 14294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:07.721081 14294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:07.721407 14294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:07.721558 14294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:07.725567 14294 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:39781
I20260929 11:03:07.725606 14360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:39781 every 8 connection(s)
I20260929 11:03:07.725975 14294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:03:07.726964 14361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:07.729250 14361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb803e86a6df419d956b9c4bc2879160: Bootstrap starting.
I20260929 11:03:07.729772 14361 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb803e86a6df419d956b9c4bc2879160: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:07.730016 14361 log.cc:824] T 00000000000000000000000000000000 P fb803e86a6df419d956b9c4bc2879160: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:07.730710 14361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb803e86a6df419d956b9c4bc2879160: No bootstrap required, opened a new log
I20260929 11:03:07.732292 14361 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb803e86a6df419d956b9c4bc2879160 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb803e86a6df419d956b9c4bc2879160" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 39781 } }
I20260929 11:03:07.732425 14361 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb803e86a6df419d956b9c4bc2879160 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:07.732447 14361 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb803e86a6df419d956b9c4bc2879160 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb803e86a6df419d956b9c4bc2879160, State: Initialized, Role: FOLLOWER
I20260929 11:03:07.732578 14361 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb803e86a6df419d956b9c4bc2879160 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb803e86a6df419d956b9c4bc2879160" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 39781 } }
I20260929 11:03:07.732658 14361 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb803e86a6df419d956b9c4bc2879160 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 11:03:07.732681 14361 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb803e86a6df419d956b9c4bc2879160 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 11:03:07.732710 14361 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb803e86a6df419d956b9c4bc2879160 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:07.733165 14361 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb803e86a6df419d956b9c4bc2879160 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb803e86a6df419d956b9c4bc2879160" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 39781 } }
I20260929 11:03:07.733253 14361 leader_election.cc:302] T 00000000000000000000000000000000 P fb803e86a6df419d956b9c4bc2879160 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb803e86a6df419d956b9c4bc2879160; no voters:
I20260929 11:03:07.733376 14361 leader_election.cc:288] T 00000000000000000000000000000000 P fb803e86a6df419d956b9c4bc2879160 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 11:03:07.733439 14364 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb803e86a6df419d956b9c4bc2879160 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:03:07.733629 14364 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb803e86a6df419d956b9c4bc2879160 [term 1 LEADER]: Becoming Leader. State: Replica: fb803e86a6df419d956b9c4bc2879160, State: Running, Role: LEADER
I20260929 11:03:07.733646 14361 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb803e86a6df419d956b9c4bc2879160 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:03:07.733733 14364 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb803e86a6df419d956b9c4bc2879160 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb803e86a6df419d956b9c4bc2879160" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 39781 } }
I20260929 11:03:07.733943 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 14294
I20260929 11:03:07.734073 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 11:03:07.734323 27107 external_mini_cluster.cc:1428] Setting key 29c721170d64940dc4fc3afeeaa5d3a8
I20260929 11:03:07.734321 14365 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb803e86a6df419d956b9c4bc2879160 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb803e86a6df419d956b9c4bc2879160" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb803e86a6df419d956b9c4bc2879160" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 39781 } } }
I20260929 11:03:07.734397 14365 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb803e86a6df419d956b9c4bc2879160 [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:07.734537 14366 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb803e86a6df419d956b9c4bc2879160 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb803e86a6df419d956b9c4bc2879160. Latest consensus state: current_term: 1 leader_uuid: "fb803e86a6df419d956b9c4bc2879160" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb803e86a6df419d956b9c4bc2879160" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 39781 } } }
I20260929 11:03:07.734618 14366 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb803e86a6df419d956b9c4bc2879160 [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:07.735117 14373 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:03:07.735618 14373 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:03:07.737128 14373 catalog_manager.cc:1347] Generated new cluster ID: 15d763a03ac0423a84a6138802c33d84
I20260929 11:03:07.737195 14373 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 11:03:07.740056 14376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:07.734818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35670 (local address 127.26.120.254:39781)
0929 11:03:07.734998 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:07.735007 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:07.735457 (+ 450us) server_negotiation.cc:407] Connection header received
0929 11:03:07.735629 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:07.735637 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:07.735718 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:07.735878 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:07.736089 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.737367 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.738238 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.738633 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.738784 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:07.739455 (+ 671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:07.739459 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:07.739459 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:07.739624 (+ 165us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:07.739654 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:07.739664 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:07.739748 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:07.739820 (+ 72us) server_negotiation.cc:299] Negotiation successful
0929 11:03:07.739896 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":78,"threads_started":1}
I20260929 11:03:07.753415 14373 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 11:03:07.753935 14373 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:03:07.757408 14373 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb803e86a6df419d956b9c4bc2879160: Generated new TSK 0
I20260929 11:03:07.757586 14373 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 11:03:07.761269 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.193:0
--local_ip_for_outbound_sockets=127.26.120.193
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:39781
--builtin_ntp_servers=127.26.120.212:41397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 11:03:07.841897 14385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:07.842150 14385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:07.842190 14385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:07.843329 14385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:07.843392 14385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:07.843434 14385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.193
I20260929 11:03:07.844774 14385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:41397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.120.193
--webserver_port=0
--tserver_master_addrs=127.26.120.254:39781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.193
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:07.845037 14385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:07.845326 14385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:07.850906 14390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:07.851122 14393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:07.851092 14385 server_base.cc:1034] running on GCE node
W20260929 11:03:07.850922 14391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:07.851364 14385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:07.851625 14385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:07.852777 14385 hybrid_clock.cc:648] HybridClock initialized: now 1790679787852763 us; error 28 us; skew 500 ppm
I20260929 11:03:07.854043 14385 webserver.cc:460] Webserver started at http://127.26.120.193:37553/ using document root <none> and password file <none>
I20260929 11:03:07.854338 14385 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:07.854383 14385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:07.854511 14385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:07.855598 14385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-0/data/instance:
uuid: "80062231cf8d4e8191881b890abd1f63"
format_stamp: "Formatted at 2026-09-29 11:03:07 on dist-test-slave-2lzq"
server_key: "438f7d019298ca8e32403cc37938f2ec"
server_key_iv: "256a05101ff4c7022849d911745a4c10"
server_key_version: "encryptionkey@0"
I20260929 11:03:07.855962 14385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance:
uuid: "80062231cf8d4e8191881b890abd1f63"
format_stamp: "Formatted at 2026-09-29 11:03:07 on dist-test-slave-2lzq"
server_key: "438f7d019298ca8e32403cc37938f2ec"
server_key_iv: "256a05101ff4c7022849d911745a4c10"
server_key_version: "encryptionkey@0"
I20260929 11:03:07.857263 14385 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 11:03:07.858027 14399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:07.858222 14385 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 11:03:07.858330 14385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-0/wal
uuid: "80062231cf8d4e8191881b890abd1f63"
format_stamp: "Formatted at 2026-09-29 11:03:07 on dist-test-slave-2lzq"
server_key: "438f7d019298ca8e32403cc37938f2ec"
server_key_iv: "256a05101ff4c7022849d911745a4c10"
server_key_version: "encryptionkey@0"
I20260929 11:03:07.858419 14385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:07.869908 14385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:07.870244 14385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:07.870398 14385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:07.870612 14385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:03:07.870939 14385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:03:07.871001 14385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:07.871042 14385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:03:07.871068 14385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:07.876858 14385 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.193:42141
I20260929 11:03:07.876892 14512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.193:42141 every 8 connection(s)
I20260929 11:03:07.877230 14385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-0/data/info.pb
I20260929 11:03:07.882139 14376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:07.877907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.193:36299 (local address 127.26.120.254:39781)
0929 11:03:07.877988 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:07.877991 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:07.878224 (+ 233us) server_negotiation.cc:407] Connection header received
0929 11:03:07.878680 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:07.878681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:07.878693 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:07.878746 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:07.879345 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.879819 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.880717 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.880906 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.880955 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:07.881817 (+ 862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:07.881818 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:07.881819 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:07.881823 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:07.881836 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:07.881839 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:07.881889 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:07.881971 (+ 82us) server_negotiation.cc:299] Negotiation successful
0929 11:03:07.881997 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 11:03:07.882292 14515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:07.877877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39781 (local address 127.26.120.193:36299)
0929 11:03:07.878097 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:07.878139 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:07.878603 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:07.878795 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:07.878798 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:07.878806 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:03:07.879279 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:07.879283 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.879894 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.879894 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:07.880646 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:07.880648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:07.880976 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:07.880977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:07.881079 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:07.881681 (+ 602us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 11:03:07.881682 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:03:07.881719 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 11:03:07.881742 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:03:07.881904 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:03:07.881909 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:03:07.881911 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 11:03:07.881962 (+ 51us) client_negotiation.cc:240] Negotiation successful
0929 11:03:07.882037 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":100,"threads_started":1}
I20260929 11:03:07.882772 14513 heartbeater.cc:347] Connected to a master server at 127.26.120.254:39781
I20260929 11:03:07.882871 14513 heartbeater.cc:464] Registering TS with master...
I20260929 11:03:07.883046 14513 heartbeater.cc:511] Master 127.26.120.254:39781 requested a full tablet report, sending...
I20260929 11:03:07.883544 14325 ts_manager.cc:194] Registered new tserver with Master: 80062231cf8d4e8191881b890abd1f63 (127.26.120.193:42141)
I20260929 11:03:07.884153 14325 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.120.193:36299
I20260929 11:03:07.886322 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 14385
I20260929 11:03:07.886415 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-0/wal/instance
I20260929 11:03:07.886566 27107 external_mini_cluster.cc:1428] Setting key 69a5572bb8b2e0a4186a16e95312d8c6
I20260929 11:03:07.887439 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.194:0
--local_ip_for_outbound_sockets=127.26.120.194
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:39781
--builtin_ntp_servers=127.26.120.212:41397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 11:03:07.967866 14516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:07.968065 14516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:07.968084 14516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:07.969136 14516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:07.969169 14516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:07.969193 14516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.194
I20260929 11:03:07.970521 14516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:41397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.120.194
--webserver_port=0
--tserver_master_addrs=127.26.120.254:39781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.194
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:07.970702 14516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:07.970938 14516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:07.976178 14524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:07.976163 14522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:07.976194 14521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:07.976722 14516 server_base.cc:1034] running on GCE node
I20260929 11:03:07.976990 14516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:07.977286 14516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:07.978461 14516 hybrid_clock.cc:648] HybridClock initialized: now 1790679787978443 us; error 34 us; skew 500 ppm
I20260929 11:03:07.979687 14516 webserver.cc:460] Webserver started at http://127.26.120.194:37331/ using document root <none> and password file <none>
I20260929 11:03:07.979933 14516 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:07.980002 14516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:07.980163 14516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:07.981249 14516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-1/data/instance:
uuid: "31f11f7625cf42449d2609eb20a2aace"
format_stamp: "Formatted at 2026-09-29 11:03:07 on dist-test-slave-2lzq"
server_key: "5073ce7f3647ca742bf7aba7e3432a00"
server_key_iv: "8316c20c5a933d2ba7abd098f4200bb4"
server_key_version: "encryptionkey@0"
I20260929 11:03:07.981627 14516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance:
uuid: "31f11f7625cf42449d2609eb20a2aace"
format_stamp: "Formatted at 2026-09-29 11:03:07 on dist-test-slave-2lzq"
server_key: "5073ce7f3647ca742bf7aba7e3432a00"
server_key_iv: "8316c20c5a933d2ba7abd098f4200bb4"
server_key_version: "encryptionkey@0"
I20260929 11:03:07.983073 14516 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 11:03:07.983903 14530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:07.984110 14516 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 11:03:07.984207 14516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-1/wal
uuid: "31f11f7625cf42449d2609eb20a2aace"
format_stamp: "Formatted at 2026-09-29 11:03:07 on dist-test-slave-2lzq"
server_key: "5073ce7f3647ca742bf7aba7e3432a00"
server_key_iv: "8316c20c5a933d2ba7abd098f4200bb4"
server_key_version: "encryptionkey@0"
I20260929 11:03:07.984299 14516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:08.007889 14516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:08.008229 14516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:08.008394 14516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:08.008662 14516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:03:08.009085 14516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:03:08.009140 14516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:08.009182 14516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:03:08.009209 14516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:08.015204 14516 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.194:33475
I20260929 11:03:08.015249 14643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.194:33475 every 8 connection(s)
I20260929 11:03:08.015614 14516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-1/data/info.pb
I20260929 11:03:08.020556 14376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:08.016349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:41495 (local address 127.26.120.254:39781)
0929 11:03:08.016397 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:08.016399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:08.016651 (+ 252us) server_negotiation.cc:407] Connection header received
0929 11:03:08.017108 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:08.017109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:08.017120 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:08.017184 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:08.017814 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:08.018367 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:08.019354 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:08.019534 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:08.019584 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:08.020339 (+ 755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:08.020340 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:08.020340 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:08.020345 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:08.020357 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:08.020360 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:08.020402 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:08.020476 (+ 74us) server_negotiation.cc:299] Negotiation successful
0929 11:03:08.020498 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 11:03:08.020800 14646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:08.016361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39781 (local address 127.26.120.194:41495)
0929 11:03:08.016530 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:08.016562 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:08.017025 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:08.017238 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:08.017242 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:08.017250 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:03:08.017743 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:08.017748 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:08.018456 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:08.018457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:08.019283 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:08.019286 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:08.019607 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:08.019608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:08.019715 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:08.020215 (+ 500us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 11:03:08.020215 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:03:08.020244 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 11:03:08.020267 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:03:08.020411 (+ 144us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:03:08.020415 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:03:08.020417 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 11:03:08.020469 (+ 52us) client_negotiation.cc:240] Negotiation successful
0929 11:03:08.020544 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":87,"threads_started":1}
I20260929 11:03:08.021330 14644 heartbeater.cc:347] Connected to a master server at 127.26.120.254:39781
I20260929 11:03:08.021425 14644 heartbeater.cc:464] Registering TS with master...
I20260929 11:03:08.021592 14644 heartbeater.cc:511] Master 127.26.120.254:39781 requested a full tablet report, sending...
I20260929 11:03:08.021937 14325 ts_manager.cc:194] Registered new tserver with Master: 31f11f7625cf42449d2609eb20a2aace (127.26.120.194:33475)
I20260929 11:03:08.022188 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 14516
I20260929 11:03:08.022300 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-1/wal/instance
I20260929 11:03:08.022396 14325 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.120.194:41495
I20260929 11:03:08.022518 27107 external_mini_cluster.cc:1428] Setting key 7a59e4551c6de05e01dd818dc969002a
I20260929 11:03:08.023408 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.120.195:0
--local_ip_for_outbound_sockets=127.26.120.195
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:39781
--builtin_ntp_servers=127.26.120.212:41397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 11:03:08.104506 14647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:08.104717 14647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:08.104738 14647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:08.105805 14647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:08.105835 14647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:08.105859 14647 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.120.195
I20260929 11:03:08.107201 14647 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:41397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.120.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.120.195
--webserver_port=0
--tserver_master_addrs=127.26.120.254:39781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14647
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.120.195
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:08.107398 14647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:08.107748 14647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:08.112892 14653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:08.112958 14647 server_base.cc:1034] running on GCE node
W20260929 11:03:08.112901 14652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:08.112911 14655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:08.113291 14647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:08.113557 14647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:08.114748 14647 hybrid_clock.cc:648] HybridClock initialized: now 1790679788114723 us; error 55 us; skew 500 ppm
I20260929 11:03:08.116216 14647 webserver.cc:460] Webserver started at http://127.26.120.195:32817/ using document root <none> and password file <none>
I20260929 11:03:08.116482 14647 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:08.116557 14647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:08.116717 14647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:08.117877 14647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-2/data/instance:
uuid: "853c77d977b841c5986cd02ae3b514d7"
format_stamp: "Formatted at 2026-09-29 11:03:08 on dist-test-slave-2lzq"
server_key: "248a57adb7cad412944aea55e8e3a197"
server_key_iv: "61a49a139e5fc428b88fd78d539905fa"
server_key_version: "encryptionkey@0"
I20260929 11:03:08.118304 14647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance:
uuid: "853c77d977b841c5986cd02ae3b514d7"
format_stamp: "Formatted at 2026-09-29 11:03:08 on dist-test-slave-2lzq"
server_key: "248a57adb7cad412944aea55e8e3a197"
server_key_iv: "61a49a139e5fc428b88fd78d539905fa"
server_key_version: "encryptionkey@0"
I20260929 11:03:08.119557 14647 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:03:08.120204 14661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:08.120383 14647 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 11:03:08.120453 14647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-2/wal
uuid: "853c77d977b841c5986cd02ae3b514d7"
format_stamp: "Formatted at 2026-09-29 11:03:08 on dist-test-slave-2lzq"
server_key: "248a57adb7cad412944aea55e8e3a197"
server_key_iv: "61a49a139e5fc428b88fd78d539905fa"
server_key_version: "encryptionkey@0"
I20260929 11:03:08.120507 14647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:08.135138 14647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:08.135468 14647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:08.135571 14647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:08.135763 14647 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 11:03:08.136099 14647 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 11:03:08.136132 14647 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:08.136194 14647 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 11:03:08.136240 14647 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:08.142091 14647 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.195:33827
I20260929 11:03:08.142119 14774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.195:33827 every 8 connection(s)
I20260929 11:03:08.142486 14647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-2/data/info.pb
I20260929 11:03:08.147519 14376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:08.143198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.195:47393 (local address 127.26.120.254:39781)
0929 11:03:08.143285 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:08.143287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:08.143496 (+ 209us) server_negotiation.cc:407] Connection header received
0929 11:03:08.144004 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:08.144006 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:08.144017 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:08.144074 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:08.144833 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:08.145352 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:08.146304 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:08.146497 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:08.146550 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:08.147280 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:08.147281 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:08.147281 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:08.147285 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:08.147298 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:08.147301 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:08.147358 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:08.147439 (+ 81us) server_negotiation.cc:299] Negotiation successful
0929 11:03:08.147462 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 11:03:08.147771 14777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:08.143179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:39781 (local address 127.26.120.195:47393)
0929 11:03:08.143389 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:08.143423 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:08.143906 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:08.144125 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:08.144141 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:08.144149 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:03:08.144653 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:08.144657 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:08.145432 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:08.145432 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:08.146229 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:08.146231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:08.146567 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:08.146568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:08.146674 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:08.147156 (+ 482us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 11:03:08.147156 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:03:08.147184 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 11:03:08.147206 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:03:08.147369 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:03:08.147373 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:03:08.147375 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 11:03:08.147427 (+ 52us) client_negotiation.cc:240] Negotiation successful
0929 11:03:08.147506 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":115,"threads_started":1}
I20260929 11:03:08.148276 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 14647
I20260929 11:03:08.148294 14775 heartbeater.cc:347] Connected to a master server at 127.26.120.254:39781
I20260929 11:03:08.148396 14775 heartbeater.cc:464] Registering TS with master...
I20260929 11:03:08.148411 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-2/wal/instance
I20260929 11:03:08.148574 14775 heartbeater.cc:511] Master 127.26.120.254:39781 requested a full tablet report, sending...
I20260929 11:03:08.148598 27107 external_mini_cluster.cc:1428] Setting key 0ea07d879de0fe38be60c07fc2c98bbd
I20260929 11:03:08.149034 14324 ts_manager.cc:194] Registered new tserver with Master: 853c77d977b841c5986cd02ae3b514d7 (127.26.120.195:33827)
I20260929 11:03:08.149435 27107 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 11:03:08.149508 14324 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.120.195:47393
I20260929 11:03:08.153280 14376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:08.150833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35682 (local address 127.26.120.254:39781)
0929 11:03:08.150867 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:08.150869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:08.150972 (+ 103us) server_negotiation.cc:407] Connection header received
0929 11:03:08.151038 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:08.151038 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:08.151046 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:08.151091 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:08.151210 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:08.151634 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:08.152353 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:08.152523 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:08.152575 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:08.153117 (+ 542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:08.153118 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:08.153119 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:08.153122 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:08.153133 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:08.153137 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:08.153176 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:08.153209 (+ 33us) server_negotiation.cc:299] Negotiation successful
0929 11:03:08.153228 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260929 11:03:08.154883 14324 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35682:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 11:03:08.156356 14324 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 11:03:08.162786 14786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:08.158602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33750 (local address 127.26.120.193:42141)
0929 11:03:08.158770 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:08.158775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:08.159029 (+ 254us) server_negotiation.cc:407] Connection header received
0929 11:03:08.159308 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:08.159312 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:08.159329 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:08.159402 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:03:08.159583 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:08.160673 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:08.161744 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:08.162389 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:08.162474 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:08.162560 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:08.162664 (+ 104us) server_negotiation.cc:299] Negotiation successful
0929 11:03:08.162703 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":84,"threads_started":1}
I20260929 11:03:08.162815 14784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:08.158535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:33475 (local address 127.0.0.1:33032)
0929 11:03:08.158959 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:08.158971 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:08.159036 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:08.159289 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:08.159290 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:08.159292 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:03:08.159326 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:08.159327 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:08.160295 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:08.160296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:08.161534 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:08.161537 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:08.162588 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:08.162588 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:08.162637 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:08.162641 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 11:03:08.162687 (+ 46us) client_negotiation.cc:240] Negotiation successful
0929 11:03:08.162724 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":101,"threads_started":1}
I20260929 11:03:08.162817 14785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:08.158478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:33827 (local address 127.0.0.1:40618)
0929 11:03:08.158685 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:08.158725 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:08.158789 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:08.159136 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:08.159139 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:08.159141 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:03:08.159208 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:08.159210 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:08.160586 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:08.160587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:08.161670 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:08.161672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:08.162554 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:08.162556 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:08.162618 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:08.162622 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 11:03:08.162654 (+ 32us) client_negotiation.cc:240] Negotiation successful
0929 11:03:08.162724 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":20,"thread_start_us":50,"threads_started":1}
I20260929 11:03:08.162946 14788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:08.158798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40618 (local address 127.26.120.195:33827)
0929 11:03:08.158942 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:08.158946 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:08.158957 (+ 11us) server_negotiation.cc:407] Connection header received
0929 11:03:08.159001 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:08.159003 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:08.159018 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:08.159127 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:03:08.159381 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:08.160503 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:08.161765 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:08.162464 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:08.162702 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:08.162798 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:08.162823 (+ 25us) server_negotiation.cc:299] Negotiation successful
0929 11:03:08.162863 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":75,"threads_started":1}
I20260929 11:03:08.163092 14783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:08.158478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:42141 (local address 127.0.0.1:33750)
0929 11:03:08.158958 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:08.158971 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:08.159036 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:08.159470 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:08.159471 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:08.159473 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:03:08.159502 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:08.159503 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:08.160757 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:08.160758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:08.161659 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:08.161662 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:08.162485 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:08.162486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:08.162564 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:08.162579 (+ 15us) client_negotiation.cc:769] Sending connection context
0929 11:03:08.162650 (+ 71us) client_negotiation.cc:240] Negotiation successful
0929 11:03:08.163013 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"mutex_wait_us":2,"thread_start_us":79,"threads_started":1}
I20260929 11:03:08.163147 14787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:08.158773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33032 (local address 127.26.120.194:33475)
0929 11:03:08.158954 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:08.158957 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:08.159053 (+ 96us) server_negotiation.cc:407] Connection header received
0929 11:03:08.159098 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:08.159100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:08.159180 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:08.159258 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:03:08.159410 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:08.160165 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:08.161623 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:08.162392 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:08.162466 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:08.162560 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:08.163017 (+ 457us) server_negotiation.cc:299] Negotiation successful
0929 11:03:08.163070 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":86,"threads_started":1}
I20260929 11:03:08.163566 14709 tablet_service.cc:1467] Processing CreateTablet for tablet ef9f92cd20424fe3a3893c32f18733ac (DEFAULT_TABLE table=test-table [id=1bd278c47f7b40939fdf8fc051385214]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:03:08.163666 14578 tablet_service.cc:1467] Processing CreateTablet for tablet ef9f92cd20424fe3a3893c32f18733ac (DEFAULT_TABLE table=test-table [id=1bd278c47f7b40939fdf8fc051385214]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:03:08.163945 14578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef9f92cd20424fe3a3893c32f18733ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:08.163945 14709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef9f92cd20424fe3a3893c32f18733ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:08.164217 14447 tablet_service.cc:1467] Processing CreateTablet for tablet ef9f92cd20424fe3a3893c32f18733ac (DEFAULT_TABLE table=test-table [id=1bd278c47f7b40939fdf8fc051385214]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 11:03:08.164522 14447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef9f92cd20424fe3a3893c32f18733ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:08.167153 14789 tablet_bootstrap.cc:492] T ef9f92cd20424fe3a3893c32f18733ac P 80062231cf8d4e8191881b890abd1f63: Bootstrap starting.
I20260929 11:03:08.167505 14791 tablet_bootstrap.cc:492] T ef9f92cd20424fe3a3893c32f18733ac P 853c77d977b841c5986cd02ae3b514d7: Bootstrap starting.
I20260929 11:03:08.167836 14789 tablet_bootstrap.cc:654] T ef9f92cd20424fe3a3893c32f18733ac P 80062231cf8d4e8191881b890abd1f63: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:08.168075 14791 tablet_bootstrap.cc:654] T ef9f92cd20424fe3a3893c32f18733ac P 853c77d977b841c5986cd02ae3b514d7: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:08.168154 14789 log.cc:824] T ef9f92cd20424fe3a3893c32f18733ac P 80062231cf8d4e8191881b890abd1f63: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:08.168368 14791 log.cc:824] T ef9f92cd20424fe3a3893c32f18733ac P 853c77d977b841c5986cd02ae3b514d7: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:08.168708 14790 tablet_bootstrap.cc:492] T ef9f92cd20424fe3a3893c32f18733ac P 31f11f7625cf42449d2609eb20a2aace: Bootstrap starting.
I20260929 11:03:08.168911 14789 tablet_bootstrap.cc:492] T ef9f92cd20424fe3a3893c32f18733ac P 80062231cf8d4e8191881b890abd1f63: No bootstrap required, opened a new log
I20260929 11:03:08.168982 14789 ts_tablet_manager.cc:1397] T ef9f92cd20424fe3a3893c32f18733ac P 80062231cf8d4e8191881b890abd1f63: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 11:03:08.169150 14791 tablet_bootstrap.cc:492] T ef9f92cd20424fe3a3893c32f18733ac P 853c77d977b841c5986cd02ae3b514d7: No bootstrap required, opened a new log
I20260929 11:03:08.169214 14791 ts_tablet_manager.cc:1397] T ef9f92cd20424fe3a3893c32f18733ac P 853c77d977b841c5986cd02ae3b514d7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 11:03:08.169353 14790 tablet_bootstrap.cc:654] T ef9f92cd20424fe3a3893c32f18733ac P 31f11f7625cf42449d2609eb20a2aace: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:08.169698 14790 log.cc:824] T ef9f92cd20424fe3a3893c32f18733ac P 31f11f7625cf42449d2609eb20a2aace: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:08.170351 14790 tablet_bootstrap.cc:492] T ef9f92cd20424fe3a3893c32f18733ac P 31f11f7625cf42449d2609eb20a2aace: No bootstrap required, opened a new log
I20260929 11:03:08.170428 14790 ts_tablet_manager.cc:1397] T ef9f92cd20424fe3a3893c32f18733ac P 31f11f7625cf42449d2609eb20a2aace: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 11:03:08.171787 14791 raft_consensus.cc:348] T ef9f92cd20424fe3a3893c32f18733ac P 853c77d977b841c5986cd02ae3b514d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80062231cf8d4e8191881b890abd1f63" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42141 } } peers { permanent_uuid: "31f11f7625cf42449d2609eb20a2aace" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 33475 } } peers { permanent_uuid: "853c77d977b841c5986cd02ae3b514d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 33827 } }
I20260929 11:03:08.171801 14789 raft_consensus.cc:348] T ef9f92cd20424fe3a3893c32f18733ac P 80062231cf8d4e8191881b890abd1f63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80062231cf8d4e8191881b890abd1f63" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42141 } } peers { permanent_uuid: "31f11f7625cf42449d2609eb20a2aace" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 33475 } } peers { permanent_uuid: "853c77d977b841c5986cd02ae3b514d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 33827 } }
I20260929 11:03:08.171957 14789 raft_consensus.cc:374] T ef9f92cd20424fe3a3893c32f18733ac P 80062231cf8d4e8191881b890abd1f63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:08.171957 14791 raft_consensus.cc:374] T ef9f92cd20424fe3a3893c32f18733ac P 853c77d977b841c5986cd02ae3b514d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:08.171993 14789 raft_consensus.cc:729] T ef9f92cd20424fe3a3893c32f18733ac P 80062231cf8d4e8191881b890abd1f63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80062231cf8d4e8191881b890abd1f63, State: Initialized, Role: FOLLOWER
I20260929 11:03:08.171993 14791 raft_consensus.cc:729] T ef9f92cd20424fe3a3893c32f18733ac P 853c77d977b841c5986cd02ae3b514d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 853c77d977b841c5986cd02ae3b514d7, State: Initialized, Role: FOLLOWER
I20260929 11:03:08.172134 14791 consensus_queue.cc:260] T ef9f92cd20424fe3a3893c32f18733ac P 853c77d977b841c5986cd02ae3b514d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80062231cf8d4e8191881b890abd1f63" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42141 } } peers { permanent_uuid: "31f11f7625cf42449d2609eb20a2aace" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 33475 } } peers { permanent_uuid: "853c77d977b841c5986cd02ae3b514d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 33827 } }
I20260929 11:03:08.172134 14789 consensus_queue.cc:260] T ef9f92cd20424fe3a3893c32f18733ac P 80062231cf8d4e8191881b890abd1f63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80062231cf8d4e8191881b890abd1f63" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42141 } } peers { permanent_uuid: "31f11f7625cf42449d2609eb20a2aace" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 33475 } } peers { permanent_uuid: "853c77d977b841c5986cd02ae3b514d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 33827 } }
I20260929 11:03:08.172392 14789 ts_tablet_manager.cc:1428] T ef9f92cd20424fe3a3893c32f18733ac P 80062231cf8d4e8191881b890abd1f63: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260929 11:03:08.172389 14791 ts_tablet_manager.cc:1428] T ef9f92cd20424fe3a3893c32f18733ac P 853c77d977b841c5986cd02ae3b514d7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 11:03:08.172343 14790 raft_consensus.cc:348] T ef9f92cd20424fe3a3893c32f18733ac P 31f11f7625cf42449d2609eb20a2aace [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80062231cf8d4e8191881b890abd1f63" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42141 } } peers { permanent_uuid: "31f11f7625cf42449d2609eb20a2aace" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 33475 } } peers { permanent_uuid: "853c77d977b841c5986cd02ae3b514d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 33827 } }
I20260929 11:03:08.172497 14790 raft_consensus.cc:374] T ef9f92cd20424fe3a3893c32f18733ac P 31f11f7625cf42449d2609eb20a2aace [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:08.172504 14775 heartbeater.cc:503] Master 127.26.120.254:39781 was elected leader, sending a full tablet report...
I20260929 11:03:08.172520 14513 heartbeater.cc:503] Master 127.26.120.254:39781 was elected leader, sending a full tablet report...
I20260929 11:03:08.172531 14790 raft_consensus.cc:729] T ef9f92cd20424fe3a3893c32f18733ac P 31f11f7625cf42449d2609eb20a2aace [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31f11f7625cf42449d2609eb20a2aace, State: Initialized, Role: FOLLOWER
I20260929 11:03:08.172654 14790 consensus_queue.cc:260] T ef9f92cd20424fe3a3893c32f18733ac P 31f11f7625cf42449d2609eb20a2aace [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80062231cf8d4e8191881b890abd1f63" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42141 } } peers { permanent_uuid: "31f11f7625cf42449d2609eb20a2aace" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 33475 } } peers { permanent_uuid: "853c77d977b841c5986cd02ae3b514d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 33827 } }
I20260929 11:03:08.173076 14790 ts_tablet_manager.cc:1428] T ef9f92cd20424fe3a3893c32f18733ac P 31f11f7625cf42449d2609eb20a2aace: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 11:03:08.173161 14644 heartbeater.cc:503] Master 127.26.120.254:39781 was elected leader, sending a full tablet report...
W20260929 11:03:08.266232 14645 tablet.cc:2369] T ef9f92cd20424fe3a3893c32f18733ac P 31f11f7625cf42449d2609eb20a2aace: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 11:03:08.377947 14514 tablet.cc:2369] T ef9f92cd20424fe3a3893c32f18733ac P 80062231cf8d4e8191881b890abd1f63: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 11:03:08.393097 14776 tablet.cc:2369] T ef9f92cd20424fe3a3893c32f18733ac P 853c77d977b841c5986cd02ae3b514d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 11:03:08.445616 14797 raft_consensus.cc:482] T ef9f92cd20424fe3a3893c32f18733ac P 31f11f7625cf42449d2609eb20a2aace [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 11:03:08.445742 14797 raft_consensus.cc:504] T ef9f92cd20424fe3a3893c32f18733ac P 31f11f7625cf42449d2609eb20a2aace [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80062231cf8d4e8191881b890abd1f63" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42141 } } peers { permanent_uuid: "31f11f7625cf42449d2609eb20a2aace" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 33475 } } peers { permanent_uuid: "853c77d977b841c5986cd02ae3b514d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 33827 } }
I20260929 11:03:08.446094 14797 leader_election.cc:288] T ef9f92cd20424fe3a3893c32f18733ac P 31f11f7625cf42449d2609eb20a2aace [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80062231cf8d4e8191881b890abd1f63 (127.26.120.193:42141), 853c77d977b841c5986cd02ae3b514d7 (127.26.120.195:33827)
I20260929 11:03:08.448800 14786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:08.446368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:41145 (local address 127.26.120.193:42141)
0929 11:03:08.446420 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:08.446421 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:08.446429 (+ 8us) server_negotiation.cc:407] Connection header received
0929 11:03:08.446458 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:08.446459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:08.446473 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:08.446540 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:03:08.446709 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:08.447191 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:08.448103 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:08.448461 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:08.448544 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:08.448552 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:08.448700 (+ 148us) server_negotiation.cc:299] Negotiation successful
0929 11:03:08.448740 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260929 11:03:08.448797 14646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:08.446245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:42141 (local address 127.26.120.194:41145)
0929 11:03:08.446304 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:08.446318 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:08.446374 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:08.446544 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:08.446545 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:08.446547 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:03:08.446632 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:08.446633 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:08.447274 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:08.447275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:08.448009 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:08.448011 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:08.448542 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:08.448543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:08.448618 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:08.448621 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 11:03:08.448683 (+ 62us) client_negotiation.cc:240] Negotiation successful
0929 11:03:08.448723 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260929 11:03:08.449080 14788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:08.446375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.120.194:58093 (local address 127.26.120.195:33827)
0929 11:03:08.446435 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:08.446437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:08.446508 (+ 71us) server_negotiation.cc:407] Connection header received
0929 11:03:08.446577 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:08.446578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:08.446593 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:08.446654 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 11:03:08.446805 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:08.447385 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:08.448309 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:08.448659 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:08.448752 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:08.448760 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:08.448955 (+ 195us) server_negotiation.cc:299] Negotiation successful
0929 11:03:08.449005 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 11:03:08.449112 14467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef9f92cd20424fe3a3893c32f18733ac" candidate_uuid: "31f11f7625cf42449d2609eb20a2aace" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80062231cf8d4e8191881b890abd1f63" is_pre_election: true
I20260929 11:03:08.449293 14798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:08.446247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.195:33827 (local address 127.26.120.194:58093)
0929 11:03:08.446431 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:08.446442 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:08.446508 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:08.446659 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:08.446660 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:08.446662 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 11:03:08.446738 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:08.446740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:08.447480 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:08.447481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:08.448212 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:08.448214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:08.448766 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:08.448767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:08.448841 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:08.448845 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 11:03:08.449177 (+ 332us) client_negotiation.cc:240] Negotiation successful
0929 11:03:08.449224 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":13,"thread_start_us":111,"threads_started":1}
I20260929 11:03:08.449295 14467 raft_consensus.cc:2454] T ef9f92cd20424fe3a3893c32f18733ac P 80062231cf8d4e8191881b890abd1f63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31f11f7625cf42449d2609eb20a2aace in term 0.
I20260929 11:03:08.449564 14531 leader_election.cc:302] T ef9f92cd20424fe3a3893c32f18733ac P 31f11f7625cf42449d2609eb20a2aace [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31f11f7625cf42449d2609eb20a2aace, 80062231cf8d4e8191881b890abd1f63; no voters:
I20260929 11:03:08.449651 14729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef9f92cd20424fe3a3893c32f18733ac" candidate_uuid: "31f11f7625cf42449d2609eb20a2aace" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "853c77d977b841c5986cd02ae3b514d7" is_pre_election: true
I20260929 11:03:08.449718 14797 raft_consensus.cc:2781] T ef9f92cd20424fe3a3893c32f18733ac P 31f11f7625cf42449d2609eb20a2aace [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 11:03:08.449808 14797 raft_consensus.cc:482] T ef9f92cd20424fe3a3893c32f18733ac P 31f11f7625cf42449d2609eb20a2aace [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 11:03:08.449811 14729 raft_consensus.cc:2454] T ef9f92cd20424fe3a3893c32f18733ac P 853c77d977b841c5986cd02ae3b514d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31f11f7625cf42449d2609eb20a2aace in term 0.
I20260929 11:03:08.449851 14797 raft_consensus.cc:3037] T ef9f92cd20424fe3a3893c32f18733ac P 31f11f7625cf42449d2609eb20a2aace [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:08.450649 14797 raft_consensus.cc:504] T ef9f92cd20424fe3a3893c32f18733ac P 31f11f7625cf42449d2609eb20a2aace [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80062231cf8d4e8191881b890abd1f63" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42141 } } peers { permanent_uuid: "31f11f7625cf42449d2609eb20a2aace" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 33475 } } peers { permanent_uuid: "853c77d977b841c5986cd02ae3b514d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 33827 } }
I20260929 11:03:08.450786 14797 leader_election.cc:288] T ef9f92cd20424fe3a3893c32f18733ac P 31f11f7625cf42449d2609eb20a2aace [CANDIDATE]: Term 1 election: Requested vote from peers 80062231cf8d4e8191881b890abd1f63 (127.26.120.193:42141), 853c77d977b841c5986cd02ae3b514d7 (127.26.120.195:33827)
I20260929 11:03:08.450911 14729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef9f92cd20424fe3a3893c32f18733ac" candidate_uuid: "31f11f7625cf42449d2609eb20a2aace" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "853c77d977b841c5986cd02ae3b514d7"
I20260929 11:03:08.450938 14467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef9f92cd20424fe3a3893c32f18733ac" candidate_uuid: "31f11f7625cf42449d2609eb20a2aace" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80062231cf8d4e8191881b890abd1f63"
I20260929 11:03:08.451001 14467 raft_consensus.cc:3037] T ef9f92cd20424fe3a3893c32f18733ac P 80062231cf8d4e8191881b890abd1f63 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:08.450992 14729 raft_consensus.cc:3037] T ef9f92cd20424fe3a3893c32f18733ac P 853c77d977b841c5986cd02ae3b514d7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:08.451942 14467 raft_consensus.cc:2454] T ef9f92cd20424fe3a3893c32f18733ac P 80062231cf8d4e8191881b890abd1f63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31f11f7625cf42449d2609eb20a2aace in term 1.
I20260929 11:03:08.451942 14729 raft_consensus.cc:2454] T ef9f92cd20424fe3a3893c32f18733ac P 853c77d977b841c5986cd02ae3b514d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31f11f7625cf42449d2609eb20a2aace in term 1.
I20260929 11:03:08.452126 14533 leader_election.cc:302] T ef9f92cd20424fe3a3893c32f18733ac P 31f11f7625cf42449d2609eb20a2aace [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31f11f7625cf42449d2609eb20a2aace, 853c77d977b841c5986cd02ae3b514d7; no voters:
I20260929 11:03:08.452239 14797 raft_consensus.cc:2781] T ef9f92cd20424fe3a3893c32f18733ac P 31f11f7625cf42449d2609eb20a2aace [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:03:08.452417 14797 raft_consensus.cc:686] T ef9f92cd20424fe3a3893c32f18733ac P 31f11f7625cf42449d2609eb20a2aace [term 1 LEADER]: Becoming Leader. State: Replica: 31f11f7625cf42449d2609eb20a2aace, State: Running, Role: LEADER
I20260929 11:03:08.452515 14797 consensus_queue.cc:237] T ef9f92cd20424fe3a3893c32f18733ac P 31f11f7625cf42449d2609eb20a2aace [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80062231cf8d4e8191881b890abd1f63" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42141 } } peers { permanent_uuid: "31f11f7625cf42449d2609eb20a2aace" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 33475 } } peers { permanent_uuid: "853c77d977b841c5986cd02ae3b514d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 33827 } }
I20260929 11:03:08.453346 14324 catalog_manager.cc:5495] T ef9f92cd20424fe3a3893c32f18733ac P 31f11f7625cf42449d2609eb20a2aace reported cstate change: term changed from 0 to 1, leader changed from <none> to 31f11f7625cf42449d2609eb20a2aace (127.26.120.194). New cstate: current_term: 1 leader_uuid: "31f11f7625cf42449d2609eb20a2aace" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80062231cf8d4e8191881b890abd1f63" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31f11f7625cf42449d2609eb20a2aace" member_type: VOTER last_known_addr { host: "127.26.120.194" port: 33475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "853c77d977b841c5986cd02ae3b514d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 33827 } health_report { overall_health: UNKNOWN } } }
I20260929 11:03:08.514147 14787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:08.511701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33048 (local address 127.26.120.194:33475)
0929 11:03:08.511761 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:08.511763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:08.511772 (+ 9us) server_negotiation.cc:407] Connection header received
0929 11:03:08.511826 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:08.511827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:08.511850 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:08.511933 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 11:03:08.512076 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:08.512557 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:08.513307 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:08.513529 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:08.513605 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:08.513760 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 11:03:08.513882 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 11:03:08.513981 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:08.513999 (+ 18us) server_negotiation.cc:299] Negotiation successful
0929 11:03:08.514018 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260929 11:03:08.516008 14729 raft_consensus.cc:1261] T ef9f92cd20424fe3a3893c32f18733ac P 853c77d977b841c5986cd02ae3b514d7 [term 1 FOLLOWER]: Refusing update from remote peer 31f11f7625cf42449d2609eb20a2aace: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 11:03:08.516016 14467 raft_consensus.cc:1261] T ef9f92cd20424fe3a3893c32f18733ac P 80062231cf8d4e8191881b890abd1f63 [term 1 FOLLOWER]: Refusing update from remote peer 31f11f7625cf42449d2609eb20a2aace: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 11:03:08.516394 14799 consensus_queue.cc:1035] T ef9f92cd20424fe3a3893c32f18733ac P 31f11f7625cf42449d2609eb20a2aace [LEADER]: Connected to new peer: Peer: permanent_uuid: "853c77d977b841c5986cd02ae3b514d7" member_type: VOTER last_known_addr { host: "127.26.120.195" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 11:03:08.516534 14797 consensus_queue.cc:1035] T ef9f92cd20424fe3a3893c32f18733ac P 31f11f7625cf42449d2609eb20a2aace [LEADER]: Connected to new peer: Peer: permanent_uuid: "80062231cf8d4e8191881b890abd1f63" member_type: VOTER last_known_addr { host: "127.26.120.193" port: 42141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 11:03:08.518846 14802 mvcc.cc:205] Tried to move back new op lower bound from 7334624413758513152 to 7334624413502005248. Current Snapshot: MvccSnapshot[applied={T|T < 7334624413758513152}]
I20260929 11:03:08.519208 14804 mvcc.cc:205] Tried to move back new op lower bound from 7334624413758513152 to 7334624413502005248. Current Snapshot: MvccSnapshot[applied={T|T < 7334624413758513152}]
I20260929 11:03:08.521428 14323 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35682:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 11:03:08.521518 14323 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35682:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 11:03:08.522301 14323 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fb803e86a6df419d956b9c4bc2879160: Sending DeleteTablet for 3 replicas of tablet ef9f92cd20424fe3a3893c32f18733ac
I20260929 11:03:08.522660 14709 tablet_service.cc:1514] Processing DeleteTablet for tablet ef9f92cd20424fe3a3893c32f18733ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:03:08 UTC) from {username='slave'} at 127.0.0.1:40618
I20260929 11:03:08.522660 14447 tablet_service.cc:1514] Processing DeleteTablet for tablet ef9f92cd20424fe3a3893c32f18733ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:03:08 UTC) from {username='slave'} at 127.0.0.1:33750
I20260929 11:03:08.522877 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 14385
I20260929 11:03:08.525019 14578 tablet_service.cc:1514] Processing DeleteTablet for tablet ef9f92cd20424fe3a3893c32f18733ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 11:03:08 UTC) from {username='slave'} at 127.0.0.1:33032
I20260929 11:03:08.525257 14814 tablet_replica.cc:330] T ef9f92cd20424fe3a3893c32f18733ac P 31f11f7625cf42449d2609eb20a2aace: stopping tablet replica
I20260929 11:03:08.525362 14814 raft_consensus.cc:2229] T ef9f92cd20424fe3a3893c32f18733ac P 31f11f7625cf42449d2609eb20a2aace [term 1 LEADER]: Raft consensus shutting down.
I20260929 11:03:08.525485 14814 raft_consensus.cc:2258] T ef9f92cd20424fe3a3893c32f18733ac P 31f11f7625cf42449d2609eb20a2aace [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 11:03:08.526929 14814 ts_tablet_manager.cc:1905] T ef9f92cd20424fe3a3893c32f18733ac P 31f11f7625cf42449d2609eb20a2aace: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 11:03:08.529017 14814 ts_tablet_manager.cc:1918] T ef9f92cd20424fe3a3893c32f18733ac P 31f11f7625cf42449d2609eb20a2aace: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 11:03:08.529098 14814 log.cc:1196] T ef9f92cd20424fe3a3893c32f18733ac P 31f11f7625cf42449d2609eb20a2aace: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-1/wal/wals/ef9f92cd20424fe3a3893c32f18733ac
W20260929 11:03:08.529299 14309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.120.193:42141 (error 108)
I20260929 11:03:08.529358 14814 ts_tablet_manager.cc:1939] T ef9f92cd20424fe3a3893c32f18733ac P 31f11f7625cf42449d2609eb20a2aace: Deleting consensus metadata
I20260929 11:03:08.529541 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 14516
I20260929 11:03:08.529943 14783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:08.529810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.193:42141 (local address 127.0.0.1:33754)
0929 11:03:08.529868 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:08.529908 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.193:42141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260929 11:03:08.530167 14309 catalog_manager.cc:4584] TS 80062231cf8d4e8191881b890abd1f63 (127.26.120.193:42141): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ef9f92cd20424fe3a3893c32f18733ac: Network error: Client connection negotiation failed: client connection to 127.26.120.193:42141: connect: Connection refused (error 111)
I20260929 11:03:08.530813 14812 tablet_replica.cc:330] T ef9f92cd20424fe3a3893c32f18733ac P 853c77d977b841c5986cd02ae3b514d7: stopping tablet replica
I20260929 11:03:08.530962 14812 raft_consensus.cc:2229] T ef9f92cd20424fe3a3893c32f18733ac P 853c77d977b841c5986cd02ae3b514d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 11:03:08.531096 14812 raft_consensus.cc:2258] T ef9f92cd20424fe3a3893c32f18733ac P 853c77d977b841c5986cd02ae3b514d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 11:03:08.531603 14812 ts_tablet_manager.cc:1905] T ef9f92cd20424fe3a3893c32f18733ac P 853c77d977b841c5986cd02ae3b514d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 11:03:08.532796 14812 ts_tablet_manager.cc:1918] T ef9f92cd20424fe3a3893c32f18733ac P 853c77d977b841c5986cd02ae3b514d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 11:03:08.532858 14812 log.cc:1196] T ef9f92cd20424fe3a3893c32f18733ac P 853c77d977b841c5986cd02ae3b514d7: Deleting WAL directory at /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790679549986039-27107-0/minicluster-data/ts-2/wal/wals/ef9f92cd20424fe3a3893c32f18733ac
I20260929 11:03:08.533093 14812 ts_tablet_manager.cc:1939] T ef9f92cd20424fe3a3893c32f18733ac P 853c77d977b841c5986cd02ae3b514d7: Deleting consensus metadata
I20260929 11:03:08.533514 14311 catalog_manager.cc:4841] TS 853c77d977b841c5986cd02ae3b514d7 (127.26.120.195:33827): tablet ef9f92cd20424fe3a3893c32f18733ac (table test-table [id=1bd278c47f7b40939fdf8fc051385214]) successfully deleted
I20260929 11:03:08.534852 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 14647
I20260929 11:03:08.534888 14783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:08.534752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.194:33475 (local address 127.0.0.1:33050)
0929 11:03:08.534810 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:08.534848 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.194:33475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260929 11:03:08.540397 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 14294
2026-09-29T11:03:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (957 ms)
[----------] 6 tests from EncryptionPolicyTest (8894 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 11:03:08 dist-test-slave-2lzq krb5kdc[14818](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 11:03:08 dist-test-slave-2lzq krb5kdc[14818](info): set up 2 sockets
Sep 29 11:03:08 dist-test-slave-2lzq krb5kdc[14818](info): commencing operation
krb5kdc: starting...
W20260929 11:03:09.574326 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 11:03:09 dist-test-slave-2lzq krb5kdc[14818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T11:03:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T11:03:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 11:03:09.690552 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35569
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35569
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:03:09.769878 14831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:09.770138 14831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:09.770169 14831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:09.771337 14831 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:03:09.771378 14831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:09.771394 14831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:09.771405 14831 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:03:09.771456 14831 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:03:09.772888 14831 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35569
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:09.773139 14831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:09.773373 14831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:09.778242 14839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:09.778311 14831 server_base.cc:1034] running on GCE node
W20260929 11:03:09.778270 14836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:09.778286 14837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:09.778694 14831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:09.779007 14831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:09.780170 14831 hybrid_clock.cc:648] HybridClock initialized: now 1790679789780158 us; error 28 us; skew 500 ppm
Sep 29 11:03:09 dist-test-slave-2lzq krb5kdc[14818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679789, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:03:09.785889 14831 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 11:03:09.786391 14831 webserver.cc:460] Webserver started at http://127.0.0.1:42155/ using document root <none> and password file <none>
I20260929 11:03:09.786624 14831 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:09.786692 14831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:09.786855 14831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:09.787946 14831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "8ba1bc900aa040e28ca4f207fcb313bd"
format_stamp: "Formatted at 2026-09-29 11:03:09 on dist-test-slave-2lzq"
server_key: "6ad8124dec960910cc9c1c0cce557ade"
server_key_iv: "6430757696f1874f82d4d41616e24e21"
server_key_version: "encryptionkey@0"
I20260929 11:03:09.788313 14831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "8ba1bc900aa040e28ca4f207fcb313bd"
format_stamp: "Formatted at 2026-09-29 11:03:09 on dist-test-slave-2lzq"
server_key: "6ad8124dec960910cc9c1c0cce557ade"
server_key_iv: "6430757696f1874f82d4d41616e24e21"
server_key_version: "encryptionkey@0"
I20260929 11:03:09.789635 14831 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 11:03:09.790414 14846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:09.790585 14831 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:09.790691 14831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "8ba1bc900aa040e28ca4f207fcb313bd"
format_stamp: "Formatted at 2026-09-29 11:03:09 on dist-test-slave-2lzq"
server_key: "6ad8124dec960910cc9c1c0cce557ade"
server_key_iv: "6430757696f1874f82d4d41616e24e21"
server_key_version: "encryptionkey@0"
I20260929 11:03:09.790766 14831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:09.805485 14831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:09.807363 14831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:09.807533 14831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:09.811472 14831 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35569
I20260929 11:03:09.811522 14898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35569 every 8 connection(s)
I20260929 11:03:09.811883 14831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:03:09.812736 14899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:09.814958 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 14831
I20260929 11:03:09.815054 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 11:03:09.815222 14899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ba1bc900aa040e28ca4f207fcb313bd: Bootstrap starting.
I20260929 11:03:09.815255 27107 external_mini_cluster.cc:1428] Setting key 40f23867c6bc233ae6b63626e47f50f4
I20260929 11:03:09.815949 14899 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ba1bc900aa040e28ca4f207fcb313bd: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:09.816329 14899 log.cc:824] T 00000000000000000000000000000000 P 8ba1bc900aa040e28ca4f207fcb313bd: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:09.817086 14899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ba1bc900aa040e28ca4f207fcb313bd: No bootstrap required, opened a new log
I20260929 11:03:09.819458 14899 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ba1bc900aa040e28ca4f207fcb313bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba1bc900aa040e28ca4f207fcb313bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35569 } }
I20260929 11:03:09.819593 14899 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ba1bc900aa040e28ca4f207fcb313bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:09.819615 14899 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ba1bc900aa040e28ca4f207fcb313bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ba1bc900aa040e28ca4f207fcb313bd, State: Initialized, Role: FOLLOWER
I20260929 11:03:09.819741 14899 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ba1bc900aa040e28ca4f207fcb313bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba1bc900aa040e28ca4f207fcb313bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35569 } }
I20260929 11:03:09.819828 14899 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ba1bc900aa040e28ca4f207fcb313bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 11:03:09.819895 14899 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ba1bc900aa040e28ca4f207fcb313bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 11:03:09.819947 14899 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ba1bc900aa040e28ca4f207fcb313bd [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:09.820662 14899 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ba1bc900aa040e28ca4f207fcb313bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba1bc900aa040e28ca4f207fcb313bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35569 } }
I20260929 11:03:09.820766 14899 leader_election.cc:302] T 00000000000000000000000000000000 P 8ba1bc900aa040e28ca4f207fcb313bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ba1bc900aa040e28ca4f207fcb313bd; no voters:
I20260929 11:03:09.820941 14899 leader_election.cc:288] T 00000000000000000000000000000000 P 8ba1bc900aa040e28ca4f207fcb313bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 11:03:09.820994 14904 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ba1bc900aa040e28ca4f207fcb313bd [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:03:09.821168 14904 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ba1bc900aa040e28ca4f207fcb313bd [term 1 LEADER]: Becoming Leader. State: Replica: 8ba1bc900aa040e28ca4f207fcb313bd, State: Running, Role: LEADER
I20260929 11:03:09.821278 14899 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ba1bc900aa040e28ca4f207fcb313bd [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:03:09.821269 14904 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ba1bc900aa040e28ca4f207fcb313bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba1bc900aa040e28ca4f207fcb313bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35569 } }
I20260929 11:03:09.823356 14905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ba1bc900aa040e28ca4f207fcb313bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ba1bc900aa040e28ca4f207fcb313bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba1bc900aa040e28ca4f207fcb313bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35569 } } }
I20260929 11:03:09.823413 14905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ba1bc900aa040e28ca4f207fcb313bd [sys.catalog]: This master's current role is: LEADER
Sep 29 11:03:09 dist-test-slave-2lzq krb5kdc[14818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 11:03:09.823223 14906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ba1bc900aa040e28ca4f207fcb313bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ba1bc900aa040e28ca4f207fcb313bd. Latest consensus state: current_term: 1 leader_uuid: "8ba1bc900aa040e28ca4f207fcb313bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba1bc900aa040e28ca4f207fcb313bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35569 } } }
I20260929 11:03:09.823624 14906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ba1bc900aa040e28ca4f207fcb313bd [sys.catalog]: This master's current role is: LEADER
W20260929 11:03:09.824012 14918 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8ba1bc900aa040e28ca4f207fcb313bd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 11:03:09.824079 14918 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 11:03:09.824172 14919 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:03:09.824399 14919 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:03:09.826509 14919 catalog_manager.cc:1347] Generated new cluster ID: 1208bd1ae4454cfdb0236ef23ea2539d
I20260929 11:03:09.826562 14919 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 11:03:09.827867 14902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:09.815844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56375 (local address 127.0.0.1:35569)
0929 11:03:09.816082 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:09.816092 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:09.816130 (+ 38us) server_negotiation.cc:407] Connection header received
0929 11:03:09.816743 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:09.816753 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:09.816819 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:09.816969 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:09.817662 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:09.819138 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:09.820165 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:09.820796 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:09.820913 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:09.824071 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:09.824075 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:09.824076 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:09.824087 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:09.826089 (+ 2002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:09.826642 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:09.826645 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:09.826645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:09.826748 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:09.826984 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:09.826985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:09.826985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:09.827226 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:09.827400 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:09.827527 (+ 127us) server_negotiation.cc:299] Negotiation successful
0929 11:03:09.827634 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":104,"threads_started":1}
I20260929 11:03:09.828783 27107 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 11:03:09.839385 14919 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 11:03:09.839867 14919 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:03:09.840895 14902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:09.834406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51351 (local address 127.0.0.1:35569)
0929 11:03:09.834445 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:09.834447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:09.834581 (+ 134us) server_negotiation.cc:407] Connection header received
0929 11:03:09.834653 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:09.834654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:09.834663 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:09.834706 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:09.835255 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:09.835751 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:09.836461 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:09.836696 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:09.836773 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:09.837985 (+ 1212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:09.837987 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:09.837987 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:09.837995 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:09.839618 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:09.840041 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:09.840042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:09.840042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:09.840093 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:09.840306 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:09.840307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:09.840307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:09.840437 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:09.840512 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:09.840743 (+ 231us) server_negotiation.cc:299] Negotiation successful
0929 11:03:09.840805 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 11:03:09.844769 14919 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ba1bc900aa040e28ca4f207fcb313bd: Generated new TSK 0
I20260929 11:03:09.844942 14919 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 11:03:09.850320 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 14831
W20260929 11:03:09.856207 14816 connection.cc:535] client connection to 127.0.0.1:35569 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35569): Connection reset by peer (error 104)
2026-09-29T11:03:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1310 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 11:03:09 dist-test-slave-2lzq krb5kdc[14930](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 11:03:09 dist-test-slave-2lzq krb5kdc[14930](info): set up 2 sockets
Sep 29 11:03:09 dist-test-slave-2lzq krb5kdc[14930](info): commencing operation
krb5kdc: starting...
W20260929 11:03:10.882354 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 11:03:10 dist-test-slave-2lzq krb5kdc[14930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T11:03:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T11:03:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 11:03:11.000619 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37425
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37425
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:03:11.080515 14943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:11.080713 14943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:11.080735 14943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:11.081863 14943 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:03:11.081907 14943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:11.081920 14943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:11.081933 14943 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:03:11.081985 14943 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:03:11.083446 14943 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37425
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:11.083676 14943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:11.083894 14943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:11.088831 14948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:11.088860 14951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:11.088863 14949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:11.088912 14943 server_base.cc:1034] running on GCE node
I20260929 11:03:11.089278 14943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:11.089599 14943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:11.090747 14943 hybrid_clock.cc:648] HybridClock initialized: now 1790679791090733 us; error 32 us; skew 500 ppm
Sep 29 11:03:11 dist-test-slave-2lzq krb5kdc[14930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679791, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:03:11.093319 14943 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 11:03:11.093750 14943 webserver.cc:460] Webserver started at http://127.0.0.1:41267/ using document root <none> and password file <none>
I20260929 11:03:11.093982 14943 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:11.094053 14943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:11.094241 14943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:11.095358 14943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "dc30ed8a964542aca1adc1c12c11343c"
format_stamp: "Formatted at 2026-09-29 11:03:11 on dist-test-slave-2lzq"
server_key: "cea208189c940c5cce0becb8d06ec96f"
server_key_iv: "bbced1fc8550c8e8169021bbc065e665"
server_key_version: "encryptionkey@0"
I20260929 11:03:11.095717 14943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "dc30ed8a964542aca1adc1c12c11343c"
format_stamp: "Formatted at 2026-09-29 11:03:11 on dist-test-slave-2lzq"
server_key: "cea208189c940c5cce0becb8d06ec96f"
server_key_iv: "bbced1fc8550c8e8169021bbc065e665"
server_key_version: "encryptionkey@0"
I20260929 11:03:11.097049 14943 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 11:03:11.097834 14958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:11.098022 14943 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 11:03:11.098145 14943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "dc30ed8a964542aca1adc1c12c11343c"
format_stamp: "Formatted at 2026-09-29 11:03:11 on dist-test-slave-2lzq"
server_key: "cea208189c940c5cce0becb8d06ec96f"
server_key_iv: "bbced1fc8550c8e8169021bbc065e665"
server_key_version: "encryptionkey@0"
I20260929 11:03:11.098235 14943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:11.113377 14943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:11.115278 14943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:11.115435 14943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:11.119513 14943 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37425
I20260929 11:03:11.119552 15010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37425 every 8 connection(s)
I20260929 11:03:11.119910 14943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:03:11.120738 15011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:11.123023 15011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc30ed8a964542aca1adc1c12c11343c: Bootstrap starting.
I20260929 11:03:11.123553 15011 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc30ed8a964542aca1adc1c12c11343c: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:11.123785 15011 log.cc:824] T 00000000000000000000000000000000 P dc30ed8a964542aca1adc1c12c11343c: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:11.124434 15011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc30ed8a964542aca1adc1c12c11343c: No bootstrap required, opened a new log
I20260929 11:03:11.125308 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 14943
I20260929 11:03:11.125437 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 11:03:11.125614 27107 external_mini_cluster.cc:1428] Setting key e4882232b6be2676e421c692fa44e345
I20260929 11:03:11.126456 15011 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc30ed8a964542aca1adc1c12c11343c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc30ed8a964542aca1adc1c12c11343c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37425 } }
I20260929 11:03:11.126626 15011 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc30ed8a964542aca1adc1c12c11343c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:11.126660 15011 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc30ed8a964542aca1adc1c12c11343c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc30ed8a964542aca1adc1c12c11343c, State: Initialized, Role: FOLLOWER
I20260929 11:03:11.126799 15011 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc30ed8a964542aca1adc1c12c11343c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc30ed8a964542aca1adc1c12c11343c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37425 } }
I20260929 11:03:11.126888 15011 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc30ed8a964542aca1adc1c12c11343c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 11:03:11.126924 15011 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc30ed8a964542aca1adc1c12c11343c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 11:03:11.127035 15011 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc30ed8a964542aca1adc1c12c11343c [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:11.127749 15011 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc30ed8a964542aca1adc1c12c11343c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc30ed8a964542aca1adc1c12c11343c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37425 } }
I20260929 11:03:11.127856 15011 leader_election.cc:302] T 00000000000000000000000000000000 P dc30ed8a964542aca1adc1c12c11343c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc30ed8a964542aca1adc1c12c11343c; no voters:
I20260929 11:03:11.128013 15011 leader_election.cc:288] T 00000000000000000000000000000000 P dc30ed8a964542aca1adc1c12c11343c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 11:03:11.128067 15016 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc30ed8a964542aca1adc1c12c11343c [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:03:11.128230 15016 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc30ed8a964542aca1adc1c12c11343c [term 1 LEADER]: Becoming Leader. State: Replica: dc30ed8a964542aca1adc1c12c11343c, State: Running, Role: LEADER
I20260929 11:03:11.128237 15011 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc30ed8a964542aca1adc1c12c11343c [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:03:11.128371 15016 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc30ed8a964542aca1adc1c12c11343c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc30ed8a964542aca1adc1c12c11343c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37425 } }
I20260929 11:03:11.128690 15016 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc30ed8a964542aca1adc1c12c11343c [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc30ed8a964542aca1adc1c12c11343c. Latest consensus state: current_term: 1 leader_uuid: "dc30ed8a964542aca1adc1c12c11343c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc30ed8a964542aca1adc1c12c11343c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37425 } } }
I20260929 11:03:11.128821 15016 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc30ed8a964542aca1adc1c12c11343c [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:11.129084 15024 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:03:11.129575 15024 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:03:11.129834 15017 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc30ed8a964542aca1adc1c12c11343c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc30ed8a964542aca1adc1c12c11343c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc30ed8a964542aca1adc1c12c11343c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37425 } } }
I20260929 11:03:11.130246 15017 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc30ed8a964542aca1adc1c12c11343c [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:11.131273 15024 catalog_manager.cc:1347] Generated new cluster ID: cbe340a9386a4043b9744a1e91614a77
I20260929 11:03:11.131325 15024 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 11:03:11.131888 15015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:11.126139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52793 (local address 127.0.0.1:37425)
0929 11:03:11.126394 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:11.126404 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:11.126442 (+ 38us) server_negotiation.cc:407] Connection header received
0929 11:03:11.126553 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:11.126561 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:11.126615 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:11.126761 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:11.127138 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:11.128520 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:11.129679 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:11.130293 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:11.130418 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:11.131170 (+ 752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:11.131173 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:11.131174 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:11.131388 (+ 214us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:11.131427 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:11.131443 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:11.131527 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:11.131660 (+ 133us) server_negotiation.cc:299] Negotiation successful
0929 11:03:11.131687 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":100,"threads_started":1}
I20260929 11:03:11.132706 27107 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 11:03:11.137238 15024 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 11:03:11.137899 15024 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:03:11.140280 15015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:11.134099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35833 (local address 127.0.0.1:37425)
0929 11:03:11.134131 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:11.134132 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:11.136767 (+ 2635us) server_negotiation.cc:407] Connection header received
0929 11:03:11.136988 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:11.136989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:11.137000 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:11.137075 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:11.137247 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:11.137942 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:11.138949 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:11.139287 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:11.139374 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:11.139998 (+ 624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:11.139999 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:11.139999 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:11.140003 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:11.140017 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:11.140021 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:11.140085 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:11.140176 (+ 91us) server_negotiation.cc:299] Negotiation successful
0929 11:03:11.140189 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 11:03:11.145017 15024 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc30ed8a964542aca1adc1c12c11343c: Generated new TSK 0
I20260929 11:03:11.145190 15024 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 11:03:11.150542 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 14943
W20260929 11:03:11.154863 14928 connection.cc:535] client connection to 127.0.0.1:37425 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37425): Connection reset by peer (error 104)
2026-09-29T11:03:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1298 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 11:03:11 dist-test-slave-2lzq krb5kdc[15042](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 11:03:11 dist-test-slave-2lzq krb5kdc[15042](info): set up 2 sockets
Sep 29 11:03:11 dist-test-slave-2lzq krb5kdc[15042](info): commencing operation
krb5kdc: starting...
W20260929 11:03:12.190377 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 11:03:12 dist-test-slave-2lzq krb5kdc[15042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679792, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T11:03:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T11:03:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 11:03:12.304852 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40323
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:03:12.384359 15055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:12.384617 15055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:12.384657 15055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:12.385787 15055 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:03:12.385859 15055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:12.385881 15055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:12.385903 15055 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:03:12.385932 15055 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:03:12.387457 15055 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40323
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:12.387746 15055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:12.387969 15055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:12.392928 15061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:12.392938 15060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:12.392942 15063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:12.393005 15055 server_base.cc:1034] running on GCE node
I20260929 11:03:12.393357 15055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:12.393613 15055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:12.394795 15055 hybrid_clock.cc:648] HybridClock initialized: now 1790679792394761 us; error 44 us; skew 500 ppm
Sep 29 11:03:12 dist-test-slave-2lzq krb5kdc[15042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679792, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:03:12.397274 15055 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 11:03:12.397775 15055 webserver.cc:460] Webserver started at http://127.0.0.1:32957/ using document root <none> and password file <none>
I20260929 11:03:12.398002 15055 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:12.398088 15055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:12.398242 15055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:12.399302 15055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "a655650a4d7843569e18056d3d53783f"
format_stamp: "Formatted at 2026-09-29 11:03:12 on dist-test-slave-2lzq"
server_key: "e4c80a1e91407d88c93ab8d9d55a90fb"
server_key_iv: "eec60d07a4ce63ef7b75ecdbd2ed6d90"
server_key_version: "encryptionkey@0"
I20260929 11:03:12.399680 15055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "a655650a4d7843569e18056d3d53783f"
format_stamp: "Formatted at 2026-09-29 11:03:12 on dist-test-slave-2lzq"
server_key: "e4c80a1e91407d88c93ab8d9d55a90fb"
server_key_iv: "eec60d07a4ce63ef7b75ecdbd2ed6d90"
server_key_version: "encryptionkey@0"
I20260929 11:03:12.401028 15055 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 11:03:12.401785 15070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:12.402004 15055 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 11:03:12.402129 15055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "a655650a4d7843569e18056d3d53783f"
format_stamp: "Formatted at 2026-09-29 11:03:12 on dist-test-slave-2lzq"
server_key: "e4c80a1e91407d88c93ab8d9d55a90fb"
server_key_iv: "eec60d07a4ce63ef7b75ecdbd2ed6d90"
server_key_version: "encryptionkey@0"
I20260929 11:03:12.402211 15055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:12.405462 15055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:12.407754 15055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:12.407912 15055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:12.411950 15055 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40323
I20260929 11:03:12.411980 15122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40323 every 8 connection(s)
I20260929 11:03:12.412303 15055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:03:12.413313 15123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:12.415627 15123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a655650a4d7843569e18056d3d53783f: Bootstrap starting.
I20260929 11:03:12.416157 15123 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a655650a4d7843569e18056d3d53783f: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:12.416385 15123 log.cc:824] T 00000000000000000000000000000000 P a655650a4d7843569e18056d3d53783f: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:12.417042 15123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a655650a4d7843569e18056d3d53783f: No bootstrap required, opened a new log
I20260929 11:03:12.418666 15123 raft_consensus.cc:348] T 00000000000000000000000000000000 P a655650a4d7843569e18056d3d53783f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a655650a4d7843569e18056d3d53783f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40323 } }
I20260929 11:03:12.418803 15123 raft_consensus.cc:374] T 00000000000000000000000000000000 P a655650a4d7843569e18056d3d53783f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:12.418826 15123 raft_consensus.cc:729] T 00000000000000000000000000000000 P a655650a4d7843569e18056d3d53783f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a655650a4d7843569e18056d3d53783f, State: Initialized, Role: FOLLOWER
I20260929 11:03:12.418967 15123 consensus_queue.cc:260] T 00000000000000000000000000000000 P a655650a4d7843569e18056d3d53783f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a655650a4d7843569e18056d3d53783f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40323 } }
I20260929 11:03:12.419116 15123 raft_consensus.cc:388] T 00000000000000000000000000000000 P a655650a4d7843569e18056d3d53783f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 11:03:12.419186 15123 raft_consensus.cc:482] T 00000000000000000000000000000000 P a655650a4d7843569e18056d3d53783f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 11:03:12.419236 15123 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a655650a4d7843569e18056d3d53783f [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:12.419327 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 15055
I20260929 11:03:12.419448 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 11:03:12.419649 27107 external_mini_cluster.cc:1428] Setting key cee22034bb6a57a2e31092f3ff70bad1
I20260929 11:03:12.419948 15123 raft_consensus.cc:504] T 00000000000000000000000000000000 P a655650a4d7843569e18056d3d53783f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a655650a4d7843569e18056d3d53783f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40323 } }
I20260929 11:03:12.420276 15123 leader_election.cc:302] T 00000000000000000000000000000000 P a655650a4d7843569e18056d3d53783f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a655650a4d7843569e18056d3d53783f; no voters:
I20260929 11:03:12.420451 15123 leader_election.cc:288] T 00000000000000000000000000000000 P a655650a4d7843569e18056d3d53783f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 11:03:12.420478 15128 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a655650a4d7843569e18056d3d53783f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:03:12.420643 15128 raft_consensus.cc:686] T 00000000000000000000000000000000 P a655650a4d7843569e18056d3d53783f [term 1 LEADER]: Becoming Leader. State: Replica: a655650a4d7843569e18056d3d53783f, State: Running, Role: LEADER
I20260929 11:03:12.420720 15123 sys_catalog.cc:564] T 00000000000000000000000000000000 P a655650a4d7843569e18056d3d53783f [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:03:12.420778 15128 consensus_queue.cc:237] T 00000000000000000000000000000000 P a655650a4d7843569e18056d3d53783f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a655650a4d7843569e18056d3d53783f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40323 } }
I20260929 11:03:12.421185 15128 sys_catalog.cc:455] T 00000000000000000000000000000000 P a655650a4d7843569e18056d3d53783f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a655650a4d7843569e18056d3d53783f. Latest consensus state: current_term: 1 leader_uuid: "a655650a4d7843569e18056d3d53783f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a655650a4d7843569e18056d3d53783f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40323 } } }
I20260929 11:03:12.421279 15128 sys_catalog.cc:458] T 00000000000000000000000000000000 P a655650a4d7843569e18056d3d53783f [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:12.421339 15130 sys_catalog.cc:455] T 00000000000000000000000000000000 P a655650a4d7843569e18056d3d53783f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a655650a4d7843569e18056d3d53783f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a655650a4d7843569e18056d3d53783f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40323 } } }
I20260929 11:03:12.421713 15130 sys_catalog.cc:458] T 00000000000000000000000000000000 P a655650a4d7843569e18056d3d53783f [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:12.421587 15132 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:03:12.422027 15127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:12.420111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55463 (local address 127.0.0.1:40323)
0929 11:03:12.420283 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:12.420290 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:12.420338 (+ 48us) server_negotiation.cc:407] Connection header received
0929 11:03:12.420474 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:12.420481 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:12.420522 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:12.420670 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:12.421527 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:12.421531 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:12.421531 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:12.421535 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:12.421592 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:12.421608 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:12.421681 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:12.421771 (+ 90us) server_negotiation.cc:299] Negotiation successful
0929 11:03:12.421815 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":83,"threads_started":1}
I20260929 11:03:12.422482 15132 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:03:12.423218 27107 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 11:03:12.424304 15132 catalog_manager.cc:1347] Generated new cluster ID: 82cfe7e8531d41169db03b621f0a5a4e
I20260929 11:03:12.424352 15132 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 11:03:12.431143 15132 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 11:03:12.431314 15132 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:03:12.434333 15127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:12.426381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50441 (local address 127.0.0.1:40323)
0929 11:03:12.426421 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:12.426422 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:12.430235 (+ 3813us) server_negotiation.cc:407] Connection header received
0929 11:03:12.430302 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:12.430303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:12.430309 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:12.430379 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:12.431109 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:12.431110 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:12.431111 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:12.431115 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:12.431131 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:12.431135 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:12.431173 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:12.434239 (+ 3066us) server_negotiation.cc:299] Negotiation successful
0929 11:03:12.434254 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 11:03:12.442947 15132 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a655650a4d7843569e18056d3d53783f: Generated new TSK 0
I20260929 11:03:12.443120 15132 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 11:03:12.445261 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 15055
2026-09-29T11:03:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1294 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 11:03:12 dist-test-slave-2lzq krb5kdc[15154](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 11:03:12 dist-test-slave-2lzq krb5kdc[15154](info): set up 2 sockets
Sep 29 11:03:12 dist-test-slave-2lzq krb5kdc[15154](info): commencing operation
krb5kdc: starting...
W20260929 11:03:13.474366 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 11:03:13 dist-test-slave-2lzq krb5kdc[15154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T11:03:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T11:03:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 11:03:13.592968 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46209
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:03:13.673107 15167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:13.673377 15167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:13.673420 15167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:13.674597 15167 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:03:13.674669 15167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:13.674693 15167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:13.674723 15167 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:03:13.674753 15167 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:03:13.676203 15167 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:13.676477 15167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:13.676704 15167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:13.681648 15172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:13.681654 15175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:13.681736 15173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:13.681797 15167 server_base.cc:1034] running on GCE node
I20260929 11:03:13.682092 15167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:13.682412 15167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:13.683568 15167 hybrid_clock.cc:648] HybridClock initialized: now 1790679793683563 us; error 37 us; skew 500 ppm
Sep 29 11:03:13 dist-test-slave-2lzq krb5kdc[15154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679793, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:03:13.686040 15167 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 11:03:13.686582 15167 webserver.cc:460] Webserver started at http://127.0.0.1:33559/ using document root <none> and password file <none>
I20260929 11:03:13.686806 15167 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:13.686877 15167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:13.687026 15167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:13.688060 15167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "8650e74c97ba4bff9f5f717a32d6e98b"
format_stamp: "Formatted at 2026-09-29 11:03:13 on dist-test-slave-2lzq"
server_key: "1bebfe1067c2f9f1cb24fecedc7c0f5a"
server_key_iv: "7db17c2c24f03cccab89d04cc09f3e76"
server_key_version: "encryptionkey@0"
I20260929 11:03:13.688441 15167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "8650e74c97ba4bff9f5f717a32d6e98b"
format_stamp: "Formatted at 2026-09-29 11:03:13 on dist-test-slave-2lzq"
server_key: "1bebfe1067c2f9f1cb24fecedc7c0f5a"
server_key_iv: "7db17c2c24f03cccab89d04cc09f3e76"
server_key_version: "encryptionkey@0"
I20260929 11:03:13.689702 15167 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260929 11:03:13.690455 15182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:13.690629 15167 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:13.690732 15167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "8650e74c97ba4bff9f5f717a32d6e98b"
format_stamp: "Formatted at 2026-09-29 11:03:13 on dist-test-slave-2lzq"
server_key: "1bebfe1067c2f9f1cb24fecedc7c0f5a"
server_key_iv: "7db17c2c24f03cccab89d04cc09f3e76"
server_key_version: "encryptionkey@0"
I20260929 11:03:13.690809 15167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:13.700968 15167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:13.703181 15167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:13.703348 15167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:13.707296 15167 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46209
I20260929 11:03:13.707361 15234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46209 every 8 connection(s)
I20260929 11:03:13.707686 15167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:03:13.708575 15235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:13.710911 15235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8650e74c97ba4bff9f5f717a32d6e98b: Bootstrap starting.
I20260929 11:03:13.711483 15235 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8650e74c97ba4bff9f5f717a32d6e98b: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:13.711768 15235 log.cc:824] T 00000000000000000000000000000000 P 8650e74c97ba4bff9f5f717a32d6e98b: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:13.712441 15235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8650e74c97ba4bff9f5f717a32d6e98b: No bootstrap required, opened a new log
I20260929 11:03:13.714030 15235 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8650e74c97ba4bff9f5f717a32d6e98b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8650e74c97ba4bff9f5f717a32d6e98b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46209 } }
I20260929 11:03:13.714233 15235 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8650e74c97ba4bff9f5f717a32d6e98b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:13.714260 15235 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8650e74c97ba4bff9f5f717a32d6e98b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8650e74c97ba4bff9f5f717a32d6e98b, State: Initialized, Role: FOLLOWER
I20260929 11:03:13.714425 15235 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8650e74c97ba4bff9f5f717a32d6e98b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8650e74c97ba4bff9f5f717a32d6e98b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46209 } }
I20260929 11:03:13.714483 15235 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8650e74c97ba4bff9f5f717a32d6e98b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 11:03:13.714540 15235 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8650e74c97ba4bff9f5f717a32d6e98b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 11:03:13.714591 15235 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8650e74c97ba4bff9f5f717a32d6e98b [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:13.715134 15235 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8650e74c97ba4bff9f5f717a32d6e98b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8650e74c97ba4bff9f5f717a32d6e98b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46209 } }
I20260929 11:03:13.715246 15235 leader_election.cc:302] T 00000000000000000000000000000000 P 8650e74c97ba4bff9f5f717a32d6e98b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8650e74c97ba4bff9f5f717a32d6e98b; no voters:
I20260929 11:03:13.715423 15235 leader_election.cc:288] T 00000000000000000000000000000000 P 8650e74c97ba4bff9f5f717a32d6e98b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 11:03:13.715447 15238 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8650e74c97ba4bff9f5f717a32d6e98b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:03:13.715600 15238 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8650e74c97ba4bff9f5f717a32d6e98b [term 1 LEADER]: Becoming Leader. State: Replica: 8650e74c97ba4bff9f5f717a32d6e98b, State: Running, Role: LEADER
I20260929 11:03:13.715691 15235 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8650e74c97ba4bff9f5f717a32d6e98b [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:03:13.715728 15238 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8650e74c97ba4bff9f5f717a32d6e98b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8650e74c97ba4bff9f5f717a32d6e98b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46209 } }
I20260929 11:03:13.716144 15239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8650e74c97ba4bff9f5f717a32d6e98b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8650e74c97ba4bff9f5f717a32d6e98b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8650e74c97ba4bff9f5f717a32d6e98b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46209 } } }
I20260929 11:03:13.716282 15239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8650e74c97ba4bff9f5f717a32d6e98b [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:13.716270 15240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8650e74c97ba4bff9f5f717a32d6e98b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8650e74c97ba4bff9f5f717a32d6e98b. Latest consensus state: current_term: 1 leader_uuid: "8650e74c97ba4bff9f5f717a32d6e98b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8650e74c97ba4bff9f5f717a32d6e98b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46209 } } }
I20260929 11:03:13.716475 15240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8650e74c97ba4bff9f5f717a32d6e98b [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:13.716804 15247 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:03:13.717270 15247 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:03:13.717653 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 15167
I20260929 11:03:13.717756 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 11:03:13.717895 27107 external_mini_cluster.cc:1428] Setting key 31c1d43a4de8d3dbe10ed4e4f6562570
I20260929 11:03:13.718850 15247 catalog_manager.cc:1347] Generated new cluster ID: e1e7918a362f4202b0137829c607f17a
I20260929 11:03:13.718900 15247 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 11:03:13 dist-test-slave-2lzq krb5kdc[15154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 11:03:13.729691 15257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:13.718363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35111 (local address 127.0.0.1:46209)
0929 11:03:13.718518 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:13.718528 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:13.718714 (+ 186us) server_negotiation.cc:407] Connection header received
0929 11:03:13.718916 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:13.718923 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:13.719000 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:13.719132 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:13.719914 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:13.721354 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:13.722433 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:13.723038 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:13.723124 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:13.726037 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:13.726041 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:13.726041 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:13.726074 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:13.728015 (+ 1941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:13.728429 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:13.728432 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:13.728433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:13.728539 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:13.728744 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:13.728745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:13.728745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:13.729052 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:13.729174 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:13.729302 (+ 128us) server_negotiation.cc:299] Negotiation successful
0929 11:03:13.729436 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":73,"threads_started":1}
I20260929 11:03:13.730834 15247 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 11:03:13.730922 27107 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 11:03:13.731346 15247 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:03:13.734444 15247 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8650e74c97ba4bff9f5f717a32d6e98b: Generated new TSK 0
I20260929 11:03:13.734627 15247 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 11:03:13.740926 15257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:13.732236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52793 (local address 127.0.0.1:46209)
0929 11:03:13.732268 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:13.732270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:13.734194 (+ 1924us) server_negotiation.cc:407] Connection header received
0929 11:03:13.734269 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:13.734270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:13.734293 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:13.734351 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:13.735010 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:13.735472 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:13.736533 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:13.736749 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:13.736797 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:13.738257 (+ 1460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:13.738259 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:13.738259 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:13.738268 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:13.739834 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:13.740232 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:13.740233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:13.740233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:13.740289 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:13.740492 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:13.740492 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:13.740493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:13.740593 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:13.740683 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:13.740799 (+ 116us) server_negotiation.cc:299] Negotiation successful
0929 11:03:13.740865 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260929 11:03:13.742938 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 15167
2026-09-29T11:03:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1299 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 11:03:13 dist-test-slave-2lzq krb5kdc[15266](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 11:03:13 dist-test-slave-2lzq krb5kdc[15266](info): set up 2 sockets
Sep 29 11:03:13 dist-test-slave-2lzq krb5kdc[15266](info): commencing operation
krb5kdc: starting...
W20260929 11:03:14.774326 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 11:03:14 dist-test-slave-2lzq krb5kdc[15266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T11:03:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T11:03:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 11:03:14.893206 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40497
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:03:14.974095 15279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:14.974311 15279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:14.974335 15279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:14.975557 15279 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:03:14.975605 15279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:14.975618 15279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:14.975631 15279 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:03:14.975644 15279 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:03:14.977192 15279 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:14.977464 15279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:14.977691 15279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:14.982558 15285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:14.982535 15287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:14.982538 15284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:14.982604 15279 server_base.cc:1034] running on GCE node
I20260929 11:03:14.982962 15279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:14.983201 15279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:14.984385 15279 hybrid_clock.cc:648] HybridClock initialized: now 1790679794984371 us; error 30 us; skew 500 ppm
Sep 29 11:03:14 dist-test-slave-2lzq krb5kdc[15266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679794, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:03:14.993821 15279 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 11:03:14.994354 15279 webserver.cc:460] Webserver started at http://127.0.0.1:39709/ using document root <none> and password file <none>
I20260929 11:03:14.994557 15279 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:14.994611 15279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:14.994726 15279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:14.995790 15279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "ca91c906822a40b1b2050201e91cc51c"
format_stamp: "Formatted at 2026-09-29 11:03:14 on dist-test-slave-2lzq"
server_key: "d47fb30608ea37f2b6a7f7facea59b82"
server_key_iv: "1a395c8debae1e78ca975b59a4ab09cf"
server_key_version: "encryptionkey@0"
I20260929 11:03:14.996121 15279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "ca91c906822a40b1b2050201e91cc51c"
format_stamp: "Formatted at 2026-09-29 11:03:14 on dist-test-slave-2lzq"
server_key: "d47fb30608ea37f2b6a7f7facea59b82"
server_key_iv: "1a395c8debae1e78ca975b59a4ab09cf"
server_key_version: "encryptionkey@0"
I20260929 11:03:14.997449 15279 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 11:03:14.998377 15294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:14.998558 15279 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:03:14.998649 15279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "ca91c906822a40b1b2050201e91cc51c"
format_stamp: "Formatted at 2026-09-29 11:03:14 on dist-test-slave-2lzq"
server_key: "d47fb30608ea37f2b6a7f7facea59b82"
server_key_iv: "1a395c8debae1e78ca975b59a4ab09cf"
server_key_version: "encryptionkey@0"
I20260929 11:03:14.998711 15279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:15.018903 15279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:15.020967 15279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:15.021075 15279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:15.025058 15279 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40497
I20260929 11:03:15.025143 15346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40497 every 8 connection(s)
I20260929 11:03:15.025527 15279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:03:15.026505 15347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:15.028126 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 15279
I20260929 11:03:15.028260 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 11:03:15.028453 27107 external_mini_cluster.cc:1428] Setting key fe55992c22c01dd89c8dddd0e48fb1a8
I20260929 11:03:15.029165 15347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca91c906822a40b1b2050201e91cc51c: Bootstrap starting.
I20260929 11:03:15.029775 15347 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca91c906822a40b1b2050201e91cc51c: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:15.030164 15347 log.cc:824] T 00000000000000000000000000000000 P ca91c906822a40b1b2050201e91cc51c: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:15.030881 15347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca91c906822a40b1b2050201e91cc51c: No bootstrap required, opened a new log
I20260929 11:03:15.033169 15347 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca91c906822a40b1b2050201e91cc51c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca91c906822a40b1b2050201e91cc51c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40497 } }
I20260929 11:03:15.033334 15347 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca91c906822a40b1b2050201e91cc51c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:15.033370 15347 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca91c906822a40b1b2050201e91cc51c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca91c906822a40b1b2050201e91cc51c, State: Initialized, Role: FOLLOWER
I20260929 11:03:15.033504 15347 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca91c906822a40b1b2050201e91cc51c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca91c906822a40b1b2050201e91cc51c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40497 } }
I20260929 11:03:15.033591 15347 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca91c906822a40b1b2050201e91cc51c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 11:03:15.033648 15347 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca91c906822a40b1b2050201e91cc51c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 11:03:15.033696 15347 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca91c906822a40b1b2050201e91cc51c [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:15.034317 15350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:15.028978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60001 (local address 127.0.0.1:40497)
0929 11:03:15.029154 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:15.029164 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:15.029199 (+ 35us) server_negotiation.cc:407] Connection header received
0929 11:03:15.029671 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:15.029678 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:15.029725 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:15.029873 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:15.030138 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:15.031291 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:15.032390 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:15.032771 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:15.032858 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:15.033556 (+ 698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:15.033560 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:15.033560 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:15.033790 (+ 230us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:15.033847 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:15.033858 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:15.033908 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:15.034014 (+ 106us) server_negotiation.cc:299] Negotiation successful
0929 11:03:15.034114 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
I20260929 11:03:15.034391 15347 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca91c906822a40b1b2050201e91cc51c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca91c906822a40b1b2050201e91cc51c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40497 } }
I20260929 11:03:15.034521 15347 leader_election.cc:302] T 00000000000000000000000000000000 P ca91c906822a40b1b2050201e91cc51c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca91c906822a40b1b2050201e91cc51c; no voters:
I20260929 11:03:15.034686 15347 leader_election.cc:288] T 00000000000000000000000000000000 P ca91c906822a40b1b2050201e91cc51c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 11:03:15.034745 15352 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca91c906822a40b1b2050201e91cc51c [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:03:15.034878 15352 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca91c906822a40b1b2050201e91cc51c [term 1 LEADER]: Becoming Leader. State: Replica: ca91c906822a40b1b2050201e91cc51c, State: Running, Role: LEADER
I20260929 11:03:15.034933 15347 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca91c906822a40b1b2050201e91cc51c [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:03:15.034953 15352 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca91c906822a40b1b2050201e91cc51c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca91c906822a40b1b2050201e91cc51c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40497 } }
I20260929 11:03:15.035348 15352 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca91c906822a40b1b2050201e91cc51c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca91c906822a40b1b2050201e91cc51c. Latest consensus state: current_term: 1 leader_uuid: "ca91c906822a40b1b2050201e91cc51c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca91c906822a40b1b2050201e91cc51c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40497 } } }
I20260929 11:03:15.035429 15352 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca91c906822a40b1b2050201e91cc51c [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:15.035526 27107 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 11:03:15.035928 15359 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:03:15.036432 15359 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:03:15.037000 15353 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca91c906822a40b1b2050201e91cc51c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca91c906822a40b1b2050201e91cc51c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca91c906822a40b1b2050201e91cc51c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40497 } } }
I20260929 11:03:15.037106 15353 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca91c906822a40b1b2050201e91cc51c [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:15.038162 15359 catalog_manager.cc:1347] Generated new cluster ID: 3ed0511ab0d1434984e1d5a2c6eafa81
I20260929 11:03:15.038216 15359 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 11:03:15.040482 15350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:15.037427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51417 (local address 127.0.0.1:40497)
0929 11:03:15.037463 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:15.037465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:15.037634 (+ 169us) server_negotiation.cc:407] Connection header received
0929 11:03:15.037717 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:15.037718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:15.037730 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:15.037764 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:15.037972 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:15.038589 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:15.039302 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:15.039519 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:15.039594 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:15.040257 (+ 663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:15.040258 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:15.040259 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:15.040263 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:15.040277 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:15.040281 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:15.040327 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:15.040384 (+ 57us) server_negotiation.cc:299] Negotiation successful
0929 11:03:15.040406 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260929 11:03:15.046514 15359 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 11:03:15.047199 15359 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:03:15.052201 15359 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca91c906822a40b1b2050201e91cc51c: Generated new TSK 0
I20260929 11:03:15.052373 15359 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 11:03:15.056965 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 15279
2026-09-29T11:03:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1312 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 11:03:15 dist-test-slave-2lzq krb5kdc[15378](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 11:03:15 dist-test-slave-2lzq krb5kdc[15378](info): set up 2 sockets
Sep 29 11:03:15 dist-test-slave-2lzq krb5kdc[15378](info): commencing operation
krb5kdc: starting...
W20260929 11:03:16.098376 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 11:03:16 dist-test-slave-2lzq krb5kdc[15378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679796, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T11:03:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T11:03:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 11:03:16.216166 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45365
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45365
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:03:16.296876 15391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:16.297127 15391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:16.297187 15391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:16.298435 15391 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:03:16.298511 15391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:16.298537 15391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:16.298566 15391 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:03:16.298597 15391 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:03:16.300117 15391 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45365
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:16.300405 15391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:16.300644 15391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:16.305639 15397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:16.305677 15396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:16.305668 15391 server_base.cc:1034] running on GCE node
W20260929 11:03:16.305635 15399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:16.306135 15391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:16.306474 15391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:16.307642 15391 hybrid_clock.cc:648] HybridClock initialized: now 1790679796307617 us; error 37 us; skew 500 ppm
Sep 29 11:03:16 dist-test-slave-2lzq krb5kdc[15378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679796, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:03:16.310187 15391 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 11:03:16.310674 15391 webserver.cc:460] Webserver started at http://127.0.0.1:38735/ using document root <none> and password file <none>
I20260929 11:03:16.310906 15391 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:16.310973 15391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:16.311105 15391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:16.312144 15391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "25711bd337ea43288e3a56eb74d494cc"
format_stamp: "Formatted at 2026-09-29 11:03:16 on dist-test-slave-2lzq"
server_key: "2960ab4affd252ec19c1655d66aecd21"
server_key_iv: "7601d79d481306e450770ba4fe91ec64"
server_key_version: "encryptionkey@0"
I20260929 11:03:16.312525 15391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "25711bd337ea43288e3a56eb74d494cc"
format_stamp: "Formatted at 2026-09-29 11:03:16 on dist-test-slave-2lzq"
server_key: "2960ab4affd252ec19c1655d66aecd21"
server_key_iv: "7601d79d481306e450770ba4fe91ec64"
server_key_version: "encryptionkey@0"
I20260929 11:03:16.313861 15391 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 11:03:16.314769 15406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:16.314965 15391 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 11:03:16.315064 15391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "25711bd337ea43288e3a56eb74d494cc"
format_stamp: "Formatted at 2026-09-29 11:03:16 on dist-test-slave-2lzq"
server_key: "2960ab4affd252ec19c1655d66aecd21"
server_key_iv: "7601d79d481306e450770ba4fe91ec64"
server_key_version: "encryptionkey@0"
I20260929 11:03:16.315146 15391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:16.318523 15391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:16.320851 15391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:16.321012 15391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:16.325217 15391 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45365
I20260929 11:03:16.325282 15458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45365 every 8 connection(s)
I20260929 11:03:16.325623 15391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:03:16.326690 15459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:16.329056 15459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25711bd337ea43288e3a56eb74d494cc: Bootstrap starting.
I20260929 11:03:16.329566 15459 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25711bd337ea43288e3a56eb74d494cc: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:16.329805 15459 log.cc:824] T 00000000000000000000000000000000 P 25711bd337ea43288e3a56eb74d494cc: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:16.330513 15459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25711bd337ea43288e3a56eb74d494cc: No bootstrap required, opened a new log
I20260929 11:03:16.330886 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 15391
I20260929 11:03:16.331012 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 11:03:16.331213 27107 external_mini_cluster.cc:1428] Setting key 034a8160d5f878c633eb4f774c84e70b
I20260929 11:03:16.333057 15459 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25711bd337ea43288e3a56eb74d494cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25711bd337ea43288e3a56eb74d494cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45365 } }
I20260929 11:03:16.333249 15459 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25711bd337ea43288e3a56eb74d494cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:16.333292 15459 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25711bd337ea43288e3a56eb74d494cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25711bd337ea43288e3a56eb74d494cc, State: Initialized, Role: FOLLOWER
I20260929 11:03:16.333423 15459 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25711bd337ea43288e3a56eb74d494cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25711bd337ea43288e3a56eb74d494cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45365 } }
I20260929 11:03:16.333529 15459 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25711bd337ea43288e3a56eb74d494cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 11:03:16.333570 15459 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25711bd337ea43288e3a56eb74d494cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 11:03:16.333619 15459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25711bd337ea43288e3a56eb74d494cc [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:16.333814 15463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:16.331714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59645 (local address 127.0.0.1:45365)
0929 11:03:16.331909 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:16.331916 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:16.331942 (+ 26us) server_negotiation.cc:407] Connection header received
0929 11:03:16.332469 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:16.332476 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:16.332500 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:16.332612 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:16.333225 (+ 613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:16.333228 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:16.333228 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:16.333232 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:16.333309 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:16.333350 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:16.333422 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:16.333505 (+ 83us) server_negotiation.cc:299] Negotiation successful
0929 11:03:16.333648 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":106,"threads_started":1}
I20260929 11:03:16.334558 15459 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25711bd337ea43288e3a56eb74d494cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25711bd337ea43288e3a56eb74d494cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45365 } }
I20260929 11:03:16.334672 15459 leader_election.cc:302] T 00000000000000000000000000000000 P 25711bd337ea43288e3a56eb74d494cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25711bd337ea43288e3a56eb74d494cc; no voters:
I20260929 11:03:16.334725 27107 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 11:03:16.334836 15459 leader_election.cc:288] T 00000000000000000000000000000000 P 25711bd337ea43288e3a56eb74d494cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 11:03:16.334890 15464 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25711bd337ea43288e3a56eb74d494cc [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:03:16.335063 15464 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25711bd337ea43288e3a56eb74d494cc [term 1 LEADER]: Becoming Leader. State: Replica: 25711bd337ea43288e3a56eb74d494cc, State: Running, Role: LEADER
I20260929 11:03:16.335172 15464 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25711bd337ea43288e3a56eb74d494cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25711bd337ea43288e3a56eb74d494cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45365 } }
I20260929 11:03:16.335057 15459 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25711bd337ea43288e3a56eb74d494cc [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:03:16.335564 15466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25711bd337ea43288e3a56eb74d494cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25711bd337ea43288e3a56eb74d494cc. Latest consensus state: current_term: 1 leader_uuid: "25711bd337ea43288e3a56eb74d494cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25711bd337ea43288e3a56eb74d494cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45365 } } }
I20260929 11:03:16.335697 15466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25711bd337ea43288e3a56eb74d494cc [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:16.335911 15465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25711bd337ea43288e3a56eb74d494cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25711bd337ea43288e3a56eb74d494cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25711bd337ea43288e3a56eb74d494cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45365 } } }
I20260929 11:03:16.336009 15465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25711bd337ea43288e3a56eb74d494cc [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:16.336158 15471 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:03:16.336652 15471 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:03:16.338675 15463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:16.337450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50081 (local address 127.0.0.1:45365)
0929 11:03:16.337644 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:16.337646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:16.337656 (+ 10us) server_negotiation.cc:407] Connection header received
0929 11:03:16.337674 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:16.337675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:16.337679 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:16.337740 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:16.338376 (+ 636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:16.338378 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:16.338378 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:16.338382 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:16.338398 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:16.338402 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:16.338462 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:16.338602 (+ 140us) server_negotiation.cc:299] Negotiation successful
0929 11:03:16.338615 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177}
I20260929 11:03:16.338794 15471 catalog_manager.cc:1347] Generated new cluster ID: b84f7d784a8e46e4997ad781945f3b5b
I20260929 11:03:16.338867 15471 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 11:03:16.350173 15471 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 11:03:16.350404 15471 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:03:16.355789 15471 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25711bd337ea43288e3a56eb74d494cc: Generated new TSK 0
I20260929 11:03:16.355997 15471 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 11:03:16.365351 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 15391
2026-09-29T11:03:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1308 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 11:03:16 dist-test-slave-2lzq krb5kdc[15490](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 11:03:16 dist-test-slave-2lzq krb5kdc[15490](info): set up 2 sockets
Sep 29 11:03:16 dist-test-slave-2lzq krb5kdc[15490](info): commencing operation
krb5kdc: starting...
W20260929 11:03:17.406477 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 11:03:17 dist-test-slave-2lzq krb5kdc[15490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T11:03:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T11:03:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 11:03:17.524686 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37615
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37615
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:03:17.604413 15503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:17.604655 15503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:17.604691 15503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:17.605818 15503 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:03:17.605891 15503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:17.605916 15503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:17.605938 15503 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:03:17.605966 15503 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:03:17.607450 15503 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37615
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:17.607719 15503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:17.607945 15503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:17.612794 15511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:17.612800 15509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:17.612851 15503 server_base.cc:1034] running on GCE node
W20260929 11:03:17.612829 15508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:17.613273 15503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:17.613617 15503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:17.614776 15503 hybrid_clock.cc:648] HybridClock initialized: now 1790679797614768 us; error 42 us; skew 500 ppm
Sep 29 11:03:17 dist-test-slave-2lzq krb5kdc[15490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679797, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:03:17.617198 15503 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 11:03:17.617620 15503 webserver.cc:460] Webserver started at http://127.0.0.1:43795/ using document root <none> and password file <none>
I20260929 11:03:17.617849 15503 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:17.617918 15503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:17.618104 15503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:17.619161 15503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "4ec6fd37b33244b3af001316456ddde9"
format_stamp: "Formatted at 2026-09-29 11:03:17 on dist-test-slave-2lzq"
server_key: "ae6057fb01a4fb20473b6ace9b4e6854"
server_key_iv: "348d2a55b7264b96fe380cc3074e4b98"
server_key_version: "encryptionkey@0"
I20260929 11:03:17.619544 15503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "4ec6fd37b33244b3af001316456ddde9"
format_stamp: "Formatted at 2026-09-29 11:03:17 on dist-test-slave-2lzq"
server_key: "ae6057fb01a4fb20473b6ace9b4e6854"
server_key_iv: "348d2a55b7264b96fe380cc3074e4b98"
server_key_version: "encryptionkey@0"
I20260929 11:03:17.620884 15503 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 11:03:17.621613 15518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:17.621788 15503 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 11:03:17.621882 15503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "4ec6fd37b33244b3af001316456ddde9"
format_stamp: "Formatted at 2026-09-29 11:03:17 on dist-test-slave-2lzq"
server_key: "ae6057fb01a4fb20473b6ace9b4e6854"
server_key_iv: "348d2a55b7264b96fe380cc3074e4b98"
server_key_version: "encryptionkey@0"
I20260929 11:03:17.621968 15503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:17.631817 15503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:17.634174 15503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:17.634334 15503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:17.638098 15503 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37615
I20260929 11:03:17.638172 15570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37615 every 8 connection(s)
I20260929 11:03:17.638511 15503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:03:17.639468 15571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:17.639468 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 15503
I20260929 11:03:17.639597 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 11:03:17.639838 27107 external_mini_cluster.cc:1428] Setting key 844a7dd12b8ed10a6d1140e4b164427e
I20260929 11:03:17.642782 15571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ec6fd37b33244b3af001316456ddde9: Bootstrap starting.
I20260929 11:03:17.643496 15571 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ec6fd37b33244b3af001316456ddde9: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:17.643790 15571 log.cc:824] T 00000000000000000000000000000000 P 4ec6fd37b33244b3af001316456ddde9: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:17.644553 15571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ec6fd37b33244b3af001316456ddde9: No bootstrap required, opened a new log
Sep 29 11:03:17 dist-test-slave-2lzq krb5kdc[15490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 11:03:17.647208 15571 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ec6fd37b33244b3af001316456ddde9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ec6fd37b33244b3af001316456ddde9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37615 } }
I20260929 11:03:17.647410 15571 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ec6fd37b33244b3af001316456ddde9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:17.647447 15571 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ec6fd37b33244b3af001316456ddde9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ec6fd37b33244b3af001316456ddde9, State: Initialized, Role: FOLLOWER
I20260929 11:03:17.647576 15571 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ec6fd37b33244b3af001316456ddde9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ec6fd37b33244b3af001316456ddde9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37615 } }
I20260929 11:03:17.647651 15571 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ec6fd37b33244b3af001316456ddde9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 11:03:17.647673 15571 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ec6fd37b33244b3af001316456ddde9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 11:03:17.647740 15571 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ec6fd37b33244b3af001316456ddde9 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:17.648270 15571 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ec6fd37b33244b3af001316456ddde9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ec6fd37b33244b3af001316456ddde9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37615 } }
I20260929 11:03:17.648353 15571 leader_election.cc:302] T 00000000000000000000000000000000 P 4ec6fd37b33244b3af001316456ddde9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ec6fd37b33244b3af001316456ddde9; no voters:
I20260929 11:03:17.648530 15571 leader_election.cc:288] T 00000000000000000000000000000000 P 4ec6fd37b33244b3af001316456ddde9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 11:03:17.648561 15576 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ec6fd37b33244b3af001316456ddde9 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:03:17.648667 15576 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ec6fd37b33244b3af001316456ddde9 [term 1 LEADER]: Becoming Leader. State: Replica: 4ec6fd37b33244b3af001316456ddde9, State: Running, Role: LEADER
I20260929 11:03:17.648790 15571 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ec6fd37b33244b3af001316456ddde9 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:03:17.648823 15576 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ec6fd37b33244b3af001316456ddde9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ec6fd37b33244b3af001316456ddde9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37615 } }
I20260929 11:03:17.649227 15576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ec6fd37b33244b3af001316456ddde9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ec6fd37b33244b3af001316456ddde9. Latest consensus state: current_term: 1 leader_uuid: "4ec6fd37b33244b3af001316456ddde9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ec6fd37b33244b3af001316456ddde9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37615 } } }
I20260929 11:03:17.649473 15576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ec6fd37b33244b3af001316456ddde9 [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:17.649398 15578 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ec6fd37b33244b3af001316456ddde9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ec6fd37b33244b3af001316456ddde9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ec6fd37b33244b3af001316456ddde9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37615 } } }
I20260929 11:03:17.649891 15578 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ec6fd37b33244b3af001316456ddde9 [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:17.650100 15585 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:03:17.650677 15585 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:03:17.651005 15574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:17.640360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.168:49107 (local address 127.0.0.1:37615)
0929 11:03:17.640527 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:17.640534 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:17.640559 (+ 25us) server_negotiation.cc:407] Connection header received
0929 11:03:17.640992 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:17.640998 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:17.641046 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:17.641155 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:17.641950 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:17.642801 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:17.643890 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:17.644296 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:17.644380 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:17.646956 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:17.646960 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:17.646961 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:17.646973 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:17.649138 (+ 2165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:17.649660 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:17.649663 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:17.649663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:17.649775 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:17.650075 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:17.650075 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:17.650076 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:17.650395 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:17.650578 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:17.650692 (+ 114us) server_negotiation.cc:299] Negotiation successful
0929 11:03:17.650793 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":84,"threads_started":1}
I20260929 11:03:17.651911 27107 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 11:03:17.652143 15585 catalog_manager.cc:1347] Generated new cluster ID: 7719a5d4af1e4c8d97eedc35e8e74906
I20260929 11:03:17.652194 15585 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 11:03:17.661154 15585 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 11:03:17.661837 15585 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:03:17.664470 15574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:17.653270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.168:35197 (local address 127.0.0.1:37615)
0929 11:03:17.653305 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:17.653307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:17.658223 (+ 4916us) server_negotiation.cc:407] Connection header received
0929 11:03:17.658296 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:17.658297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:17.658307 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:17.658361 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:17.658874 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:17.659344 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:17.660048 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:17.660263 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:17.660346 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:17.661674 (+ 1328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:17.661675 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:17.661676 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:17.661682 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:17.663070 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:17.663431 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:17.663432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:17.663433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:17.663490 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:17.663766 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:17.663767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:17.663768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:17.663939 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:17.664043 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:17.664248 (+ 205us) server_negotiation.cc:299] Negotiation successful
0929 11:03:17.664341 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 11:03:17.666031 15585 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ec6fd37b33244b3af001316456ddde9: Generated new TSK 0
I20260929 11:03:17.666215 15585 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 11:03:17.668767 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 15503
W20260929 11:03:17.672711 15488 connection.cc:535] client connection to 127.0.0.1:37615 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37615): Connection reset by peer (error 104)
2026-09-29T11:03:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1302 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 11:03:17 dist-test-slave-2lzq krb5kdc[15602](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 11:03:17 dist-test-slave-2lzq krb5kdc[15602](info): set up 2 sockets
Sep 29 11:03:17 dist-test-slave-2lzq krb5kdc[15602](info): commencing operation
krb5kdc: starting...
W20260929 11:03:18.718463 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 11:03:18 dist-test-slave-2lzq krb5kdc[15602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T11:03:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T11:03:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 11:03:18.836923 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44735
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44735
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:03:18.916417 15615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:18.916610 15615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:18.916631 15615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:18.917714 15615 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:03:18.917749 15615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:18.917763 15615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:18.917775 15615 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:03:18.917788 15615 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:03:18.919797 15615 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44735
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:18.919991 15615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:18.920159 15615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:18.924999 15620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:18.925011 15623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:18.925016 15621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:18.925128 15615 server_base.cc:1034] running on GCE node
I20260929 11:03:18.925463 15615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:18.925745 15615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:18.926911 15615 hybrid_clock.cc:648] HybridClock initialized: now 1790679798926894 us; error 37 us; skew 500 ppm
Sep 29 11:03:18 dist-test-slave-2lzq krb5kdc[15602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679798, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:03:18.929294 15615 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 11:03:18.929773 15615 webserver.cc:460] Webserver started at http://127.0.0.1:43235/ using document root <none> and password file <none>
I20260929 11:03:18.930012 15615 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:18.930104 15615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:18.930258 15615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:18.931357 15615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "359ee82b8d07486a9e379377698497aa"
format_stamp: "Formatted at 2026-09-29 11:03:18 on dist-test-slave-2lzq"
server_key: "4584aa3b4fc562480cd1f3a5606b3370"
server_key_iv: "f1fbc8338aeef5077df710a99b814aed"
server_key_version: "encryptionkey@0"
I20260929 11:03:18.931723 15615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "359ee82b8d07486a9e379377698497aa"
format_stamp: "Formatted at 2026-09-29 11:03:18 on dist-test-slave-2lzq"
server_key: "4584aa3b4fc562480cd1f3a5606b3370"
server_key_iv: "f1fbc8338aeef5077df710a99b814aed"
server_key_version: "encryptionkey@0"
I20260929 11:03:18.933017 15615 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:03:18.933734 15630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:18.933954 15615 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:03:18.934054 15615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "359ee82b8d07486a9e379377698497aa"
format_stamp: "Formatted at 2026-09-29 11:03:18 on dist-test-slave-2lzq"
server_key: "4584aa3b4fc562480cd1f3a5606b3370"
server_key_iv: "f1fbc8338aeef5077df710a99b814aed"
server_key_version: "encryptionkey@0"
I20260929 11:03:18.934166 15615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:18.949537 15615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:18.951383 15615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:18.951547 15615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:18.955588 15615 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44735
I20260929 11:03:18.955624 15682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44735 every 8 connection(s)
I20260929 11:03:18.955993 15615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:03:18.956807 15683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:18.959137 15683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 359ee82b8d07486a9e379377698497aa: Bootstrap starting.
I20260929 11:03:18.959699 15683 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 359ee82b8d07486a9e379377698497aa: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:18.959995 15683 log.cc:824] T 00000000000000000000000000000000 P 359ee82b8d07486a9e379377698497aa: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:18.960660 15683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 359ee82b8d07486a9e379377698497aa: No bootstrap required, opened a new log
I20260929 11:03:18.961790 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 15615
I20260929 11:03:18.961922 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 11:03:18.962144 27107 external_mini_cluster.cc:1428] Setting key 6fae801165ef486226fbd98f4a41195a
I20260929 11:03:18.962442 15683 raft_consensus.cc:348] T 00000000000000000000000000000000 P 359ee82b8d07486a9e379377698497aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "359ee82b8d07486a9e379377698497aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44735 } }
I20260929 11:03:18.962693 15683 raft_consensus.cc:374] T 00000000000000000000000000000000 P 359ee82b8d07486a9e379377698497aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:18.962767 15683 raft_consensus.cc:729] T 00000000000000000000000000000000 P 359ee82b8d07486a9e379377698497aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 359ee82b8d07486a9e379377698497aa, State: Initialized, Role: FOLLOWER
I20260929 11:03:18.962896 15683 consensus_queue.cc:260] T 00000000000000000000000000000000 P 359ee82b8d07486a9e379377698497aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "359ee82b8d07486a9e379377698497aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44735 } }
I20260929 11:03:18.962965 15683 raft_consensus.cc:388] T 00000000000000000000000000000000 P 359ee82b8d07486a9e379377698497aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 11:03:18.962985 15683 raft_consensus.cc:482] T 00000000000000000000000000000000 P 359ee82b8d07486a9e379377698497aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 11:03:18.963053 15683 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 359ee82b8d07486a9e379377698497aa [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:18.963699 15683 raft_consensus.cc:504] T 00000000000000000000000000000000 P 359ee82b8d07486a9e379377698497aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "359ee82b8d07486a9e379377698497aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44735 } }
I20260929 11:03:18.963792 15683 leader_election.cc:302] T 00000000000000000000000000000000 P 359ee82b8d07486a9e379377698497aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 359ee82b8d07486a9e379377698497aa; no voters:
I20260929 11:03:18.963958 15683 leader_election.cc:288] T 00000000000000000000000000000000 P 359ee82b8d07486a9e379377698497aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 11:03:18.963989 15688 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 359ee82b8d07486a9e379377698497aa [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:03:18.964097 15688 raft_consensus.cc:686] T 00000000000000000000000000000000 P 359ee82b8d07486a9e379377698497aa [term 1 LEADER]: Becoming Leader. State: Replica: 359ee82b8d07486a9e379377698497aa, State: Running, Role: LEADER
I20260929 11:03:18.964206 15683 sys_catalog.cc:564] T 00000000000000000000000000000000 P 359ee82b8d07486a9e379377698497aa [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:03:18.964244 15688 consensus_queue.cc:237] T 00000000000000000000000000000000 P 359ee82b8d07486a9e379377698497aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "359ee82b8d07486a9e379377698497aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44735 } }
I20260929 11:03:18.964654 15688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 359ee82b8d07486a9e379377698497aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 359ee82b8d07486a9e379377698497aa. Latest consensus state: current_term: 1 leader_uuid: "359ee82b8d07486a9e379377698497aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "359ee82b8d07486a9e379377698497aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44735 } } }
I20260929 11:03:18.964661 15689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 359ee82b8d07486a9e379377698497aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "359ee82b8d07486a9e379377698497aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "359ee82b8d07486a9e379377698497aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44735 } } }
I20260929 11:03:18.964907 15689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 359ee82b8d07486a9e379377698497aa [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:18.964818 15688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 359ee82b8d07486a9e379377698497aa [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:18.965198 15697 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:03:18.965762 15697 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:03:18.967825 15697 catalog_manager.cc:1347] Generated new cluster ID: dbd46de4405440018333f0bed2879762
I20260929 11:03:18.967907 15697 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 11:03:18.968050 15687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:18.962677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.168:35109 (local address 127.0.0.1:44735)
0929 11:03:18.962937 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:18.962946 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:18.963001 (+ 55us) server_negotiation.cc:407] Connection header received
0929 11:03:18.963123 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:18.963130 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:18.963194 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:18.963299 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:18.963577 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:18.964784 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:18.965938 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:18.966457 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:18.966577 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:18.967240 (+ 663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:18.967244 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:18.967244 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:18.967490 (+ 246us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:18.967525 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:18.967539 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:18.967625 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:18.967753 (+ 128us) server_negotiation.cc:299] Negotiation successful
0929 11:03:18.967782 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":88,"threads_started":1}
I20260929 11:03:18.968930 27107 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 11:03:18.972975 15687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:18.970370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.168:34611 (local address 127.0.0.1:44735)
0929 11:03:18.970414 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:18.970415 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:18.970424 (+ 9us) server_negotiation.cc:407] Connection header received
0929 11:03:18.970455 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:18.970456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:18.970464 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:18.970511 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:18.970611 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:18.971057 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:18.971770 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:18.971997 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:18.972071 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:18.972756 (+ 685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:18.972757 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:18.972757 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:18.972761 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:18.972774 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:18.972777 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:18.972830 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:18.972888 (+ 58us) server_negotiation.cc:299] Negotiation successful
0929 11:03:18.972902 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260929 11:03:18.990262 15697 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 11:03:18.990899 15697 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:03:18.999120 15697 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 359ee82b8d07486a9e379377698497aa: Generated new TSK 0
I20260929 11:03:18.999310 15697 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 11:03:19.005988 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 15615
W20260929 11:03:19.010140 15600 connection.cc:535] client connection to 127.0.0.1:44735 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44735): Connection reset by peer (error 104)
2026-09-29T11:03:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1337 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 11:03:19 dist-test-slave-2lzq krb5kdc[15714](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 11:03:19 dist-test-slave-2lzq krb5kdc[15714](info): set up 2 sockets
Sep 29 11:03:19 dist-test-slave-2lzq krb5kdc[15714](info): commencing operation
krb5kdc: starting...
W20260929 11:03:20.038458 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 11:03:20 dist-test-slave-2lzq krb5kdc[15714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T11:03:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T11:03:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 11:03:20.154758 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34919
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34919
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:03:20.236426 15727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:20.236626 15727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:20.236651 15727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:20.237916 15727 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:03:20.237960 15727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:20.237974 15727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:20.238009 15727 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:03:20.238034 15727 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:03:20.239604 15727 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34919
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:20.239853 15727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:20.240077 15727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:20.244922 15735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:20.244930 15732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:20.245028 15727 server_base.cc:1034] running on GCE node
W20260929 11:03:20.244935 15733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:20.245468 15727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:20.245755 15727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:20.246927 15727 hybrid_clock.cc:648] HybridClock initialized: now 1790679800246917 us; error 38 us; skew 500 ppm
Sep 29 11:03:20 dist-test-slave-2lzq krb5kdc[15714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679800, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:03:20.255337 15727 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 11:03:20.255781 15727 webserver.cc:460] Webserver started at http://127.0.0.1:35667/ using document root <none> and password file <none>
I20260929 11:03:20.256021 15727 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:20.256093 15727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:20.256246 15727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:20.257318 15727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "0c52ac23f4564e6db4b24aeef2734f5b"
format_stamp: "Formatted at 2026-09-29 11:03:20 on dist-test-slave-2lzq"
server_key: "13558e89a368214523d81da497320a46"
server_key_iv: "5c0a9b15c27823b83387b0739f240d81"
server_key_version: "encryptionkey@0"
I20260929 11:03:20.257687 15727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "0c52ac23f4564e6db4b24aeef2734f5b"
format_stamp: "Formatted at 2026-09-29 11:03:20 on dist-test-slave-2lzq"
server_key: "13558e89a368214523d81da497320a46"
server_key_iv: "5c0a9b15c27823b83387b0739f240d81"
server_key_version: "encryptionkey@0"
I20260929 11:03:20.259130 15727 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 11:03:20.259862 15742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:20.260049 15727 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:20.260155 15727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "0c52ac23f4564e6db4b24aeef2734f5b"
format_stamp: "Formatted at 2026-09-29 11:03:20 on dist-test-slave-2lzq"
server_key: "13558e89a368214523d81da497320a46"
server_key_iv: "5c0a9b15c27823b83387b0739f240d81"
server_key_version: "encryptionkey@0"
I20260929 11:03:20.260231 15727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:20.263563 15727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:20.265787 15727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:20.265945 15727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:20.269773 15727 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34919
I20260929 11:03:20.269804 15794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34919 every 8 connection(s)
I20260929 11:03:20.270191 15727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:03:20.271162 15795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:20.273388 15795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c52ac23f4564e6db4b24aeef2734f5b: Bootstrap starting.
I20260929 11:03:20.273954 15795 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c52ac23f4564e6db4b24aeef2734f5b: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:20.274272 15795 log.cc:824] T 00000000000000000000000000000000 P 0c52ac23f4564e6db4b24aeef2734f5b: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:20.274930 15795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c52ac23f4564e6db4b24aeef2734f5b: No bootstrap required, opened a new log
I20260929 11:03:20.276489 15795 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c52ac23f4564e6db4b24aeef2734f5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c52ac23f4564e6db4b24aeef2734f5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34919 } }
I20260929 11:03:20.276638 15795 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c52ac23f4564e6db4b24aeef2734f5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:20.276721 15795 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c52ac23f4564e6db4b24aeef2734f5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c52ac23f4564e6db4b24aeef2734f5b, State: Initialized, Role: FOLLOWER
I20260929 11:03:20.276919 15795 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c52ac23f4564e6db4b24aeef2734f5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c52ac23f4564e6db4b24aeef2734f5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34919 } }
I20260929 11:03:20.277004 15795 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c52ac23f4564e6db4b24aeef2734f5b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 11:03:20.277055 15795 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c52ac23f4564e6db4b24aeef2734f5b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 11:03:20.277107 15795 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c52ac23f4564e6db4b24aeef2734f5b [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:20.277657 15795 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c52ac23f4564e6db4b24aeef2734f5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c52ac23f4564e6db4b24aeef2734f5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34919 } }
I20260929 11:03:20.277769 15795 leader_election.cc:302] T 00000000000000000000000000000000 P 0c52ac23f4564e6db4b24aeef2734f5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c52ac23f4564e6db4b24aeef2734f5b; no voters:
I20260929 11:03:20.277923 15795 leader_election.cc:288] T 00000000000000000000000000000000 P 0c52ac23f4564e6db4b24aeef2734f5b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 11:03:20.277988 15798 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c52ac23f4564e6db4b24aeef2734f5b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:03:20.278164 15798 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c52ac23f4564e6db4b24aeef2734f5b [term 1 LEADER]: Becoming Leader. State: Replica: 0c52ac23f4564e6db4b24aeef2734f5b, State: Running, Role: LEADER
I20260929 11:03:20.278208 15795 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c52ac23f4564e6db4b24aeef2734f5b [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:03:20.278307 15798 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c52ac23f4564e6db4b24aeef2734f5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c52ac23f4564e6db4b24aeef2734f5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34919 } }
I20260929 11:03:20.278719 15800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c52ac23f4564e6db4b24aeef2734f5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c52ac23f4564e6db4b24aeef2734f5b. Latest consensus state: current_term: 1 leader_uuid: "0c52ac23f4564e6db4b24aeef2734f5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c52ac23f4564e6db4b24aeef2734f5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34919 } } }
I20260929 11:03:20.278717 15799 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c52ac23f4564e6db4b24aeef2734f5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c52ac23f4564e6db4b24aeef2734f5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c52ac23f4564e6db4b24aeef2734f5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34919 } } }
I20260929 11:03:20.278869 15800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c52ac23f4564e6db4b24aeef2734f5b [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:20.278962 15799 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c52ac23f4564e6db4b24aeef2734f5b [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:20.279335 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 15727
I20260929 11:03:20.279444 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 11:03:20.279588 27107 external_mini_cluster.cc:1428] Setting key 397fa4a389420b6f09f2378ebd18206c
I20260929 11:03:20.279791 15804 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:03:20.280386 15804 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:03:20.281908 15804 catalog_manager.cc:1347] Generated new cluster ID: ec51f7fcb5c0421bb12b41cf2999a4ff
I20260929 11:03:20.281961 15804 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 11:03:20.283594 15817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:20.280093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.168:50077 (local address 127.0.0.1:34919)
0929 11:03:20.282158 (+ 2065us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:20.282167 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:20.282200 (+ 33us) server_negotiation.cc:407] Connection header received
0929 11:03:20.282326 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:20.282333 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:20.282362 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:20.282507 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:20.283170 (+ 663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:20.283173 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:20.283174 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:20.283177 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:20.283223 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:20.283238 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:20.283307 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:20.283371 (+ 64us) server_negotiation.cc:299] Negotiation successful
0929 11:03:20.283417 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2004,"thread_start_us":68,"threads_started":1}
I20260929 11:03:20.284473 27107 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 11:03:20.287177 15817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:20.285964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.168:50337 (local address 127.0.0.1:34919)
0929 11:03:20.286025 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:20.286027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:20.286258 (+ 231us) server_negotiation.cc:407] Connection header received
0929 11:03:20.286315 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:20.286315 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:20.286321 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:20.286375 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:20.286947 (+ 572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:20.286948 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:20.286949 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:20.286952 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:20.286966 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:20.286970 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:20.287023 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:20.287108 (+ 85us) server_negotiation.cc:299] Negotiation successful
0929 11:03:20.287121 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 11:03:20.288714 15804 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 11:03:20.288877 15804 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:03:20.297752 15804 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c52ac23f4564e6db4b24aeef2734f5b: Generated new TSK 0
I20260929 11:03:20.297940 15804 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 11:03:20.299405 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 15727
2026-09-29T11:03:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1293 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 11:03:20 dist-test-slave-2lzq krb5kdc[15826](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 11:03:20 dist-test-slave-2lzq krb5kdc[15826](info): set up 2 sockets
Sep 29 11:03:20 dist-test-slave-2lzq krb5kdc[15826](info): commencing operation
krb5kdc: starting...
W20260929 11:03:21.350373 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 11:03:21 dist-test-slave-2lzq krb5kdc[15826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T11:03:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T11:03:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 11:03:21.469287 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46161
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:03:21.548552 15839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:21.548796 15839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:21.548835 15839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:21.549971 15839 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:03:21.550041 15839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:21.550109 15839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:21.550154 15839 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:03:21.550176 15839 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:03:21.551666 15839 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46161
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:21.551951 15839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:21.552183 15839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:21.557204 15847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:21.557231 15845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:21.557216 15844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:21.557425 15839 server_base.cc:1034] running on GCE node
I20260929 11:03:21.557745 15839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:21.557996 15839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:21.559142 15839 hybrid_clock.cc:648] HybridClock initialized: now 1790679801559095 us; error 65 us; skew 500 ppm
Sep 29 11:03:21 dist-test-slave-2lzq krb5kdc[15826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679801, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:03:21.561522 15839 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 11:03:21.561905 15839 webserver.cc:460] Webserver started at http://127.0.0.1:39327/ using document root <none> and password file <none>
I20260929 11:03:21.562160 15839 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:21.562209 15839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:21.562300 15839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:21.563289 15839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "0c562092eac7440c8c5bcd5e37187efd"
format_stamp: "Formatted at 2026-09-29 11:03:21 on dist-test-slave-2lzq"
server_key: "30773e528623f82a5c579b3593d64130"
server_key_iv: "dcf43a29dd8e753e03a57c3493cce644"
server_key_version: "encryptionkey@0"
I20260929 11:03:21.563620 15839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "0c562092eac7440c8c5bcd5e37187efd"
format_stamp: "Formatted at 2026-09-29 11:03:21 on dist-test-slave-2lzq"
server_key: "30773e528623f82a5c579b3593d64130"
server_key_iv: "dcf43a29dd8e753e03a57c3493cce644"
server_key_version: "encryptionkey@0"
I20260929 11:03:21.564869 15839 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 11:03:21.565538 15854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:21.565701 15839 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 11:03:21.565805 15839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "0c562092eac7440c8c5bcd5e37187efd"
format_stamp: "Formatted at 2026-09-29 11:03:21 on dist-test-slave-2lzq"
server_key: "30773e528623f82a5c579b3593d64130"
server_key_iv: "dcf43a29dd8e753e03a57c3493cce644"
server_key_version: "encryptionkey@0"
I20260929 11:03:21.565882 15839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:21.573966 15839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:21.575736 15839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:21.575897 15839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:21.579921 15839 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46161
I20260929 11:03:21.579953 15906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46161 every 8 connection(s)
I20260929 11:03:21.580298 15839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:03:21.581149 15907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:21.583341 15907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c562092eac7440c8c5bcd5e37187efd: Bootstrap starting.
I20260929 11:03:21.583882 15907 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c562092eac7440c8c5bcd5e37187efd: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:21.583956 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 15839
I20260929 11:03:21.584097 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 11:03:21.584206 15907 log.cc:824] T 00000000000000000000000000000000 P 0c562092eac7440c8c5bcd5e37187efd: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:21.584292 27107 external_mini_cluster.cc:1428] Setting key 1a5d1478ac09d200767db11fb9fc6b1a
I20260929 11:03:21.585052 15907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c562092eac7440c8c5bcd5e37187efd: No bootstrap required, opened a new log
I20260929 11:03:21.587590 15907 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c562092eac7440c8c5bcd5e37187efd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c562092eac7440c8c5bcd5e37187efd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46161 } }
I20260929 11:03:21.587765 15907 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c562092eac7440c8c5bcd5e37187efd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:21.587802 15907 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c562092eac7440c8c5bcd5e37187efd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c562092eac7440c8c5bcd5e37187efd, State: Initialized, Role: FOLLOWER
I20260929 11:03:21.587935 15907 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c562092eac7440c8c5bcd5e37187efd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c562092eac7440c8c5bcd5e37187efd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46161 } }
I20260929 11:03:21.588006 15907 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c562092eac7440c8c5bcd5e37187efd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 11:03:21.588032 15907 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c562092eac7440c8c5bcd5e37187efd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 11:03:21.588074 15907 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c562092eac7440c8c5bcd5e37187efd [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:21.588704 15907 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c562092eac7440c8c5bcd5e37187efd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c562092eac7440c8c5bcd5e37187efd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46161 } }
I20260929 11:03:21.588792 15907 leader_election.cc:302] T 00000000000000000000000000000000 P 0c562092eac7440c8c5bcd5e37187efd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c562092eac7440c8c5bcd5e37187efd; no voters:
I20260929 11:03:21.588984 15907 leader_election.cc:288] T 00000000000000000000000000000000 P 0c562092eac7440c8c5bcd5e37187efd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 11:03:21.589011 15912 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c562092eac7440c8c5bcd5e37187efd [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:03:21.589211 15912 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c562092eac7440c8c5bcd5e37187efd [term 1 LEADER]: Becoming Leader. State: Replica: 0c562092eac7440c8c5bcd5e37187efd, State: Running, Role: LEADER
I20260929 11:03:21.589341 15907 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c562092eac7440c8c5bcd5e37187efd [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:03:21.589355 15912 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c562092eac7440c8c5bcd5e37187efd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c562092eac7440c8c5bcd5e37187efd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46161 } }
I20260929 11:03:21.589818 15913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c562092eac7440c8c5bcd5e37187efd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c562092eac7440c8c5bcd5e37187efd. Latest consensus state: current_term: 1 leader_uuid: "0c562092eac7440c8c5bcd5e37187efd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c562092eac7440c8c5bcd5e37187efd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46161 } } }
I20260929 11:03:21.589947 15913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c562092eac7440c8c5bcd5e37187efd [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:21.590224 15921 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:03:21.590731 15921 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 29 11:03:21 dist-test-slave-2lzq krb5kdc[15826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 11:03:21.592339 15921 catalog_manager.cc:1347] Generated new cluster ID: 258b84bc027041e5a29bbd0c91112b27
I20260929 11:03:21.592355 15914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c562092eac7440c8c5bcd5e37187efd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c562092eac7440c8c5bcd5e37187efd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c562092eac7440c8c5bcd5e37187efd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46161 } } }
I20260929 11:03:21.592864 15921 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 11:03:21.592890 15914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c562092eac7440c8c5bcd5e37187efd [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:21.596391 15911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:21.584826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.168:47259 (local address 127.0.0.1:46161)
0929 11:03:21.584984 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:21.584993 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:21.585029 (+ 36us) server_negotiation.cc:407] Connection header received
0929 11:03:21.585484 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:21.585491 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:21.585537 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:21.585644 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:21.586465 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:21.587322 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:21.588399 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:21.588830 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:21.588960 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:21.591847 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:21.591851 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:21.591851 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:21.591862 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:21.594851 (+ 2989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:21.595247 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:21.595250 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:21.595251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:21.595337 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:21.595580 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:21.595580 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:21.595581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:21.595807 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:21.596022 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:21.596116 (+ 94us) server_negotiation.cc:299] Negotiation successful
0929 11:03:21.596222 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":86,"threads_started":1}
I20260929 11:03:21.597366 27107 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 11:03:21.605145 15911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:21.598789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.168:51755 (local address 127.0.0.1:46161)
0929 11:03:21.598829 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:21.598830 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:21.598974 (+ 144us) server_negotiation.cc:407] Connection header received
0929 11:03:21.599047 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:21.599048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:21.599058 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:21.599113 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:21.599614 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:21.600094 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:21.600963 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:21.601209 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:21.601290 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:21.602551 (+ 1261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:21.602553 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:21.602553 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:21.602560 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:21.603957 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:21.604326 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:21.604327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:21.604327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:21.604383 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:21.604609 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:21.604611 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:21.604611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:21.604720 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:21.604848 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:21.604970 (+ 122us) server_negotiation.cc:299] Negotiation successful
0929 11:03:21.605032 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 11:03:21.624976 15921 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 11:03:21.625705 15921 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:03:21.632334 15921 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c562092eac7440c8c5bcd5e37187efd: Generated new TSK 0
I20260929 11:03:21.632592 15921 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 11:03:21.634570 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 15839
W20260929 11:03:21.638782 15824 connection.cc:535] client connection to 127.0.0.1:46161 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46161): Connection reset by peer (error 104)
2026-09-29T11:03:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1334 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 11:03:21 dist-test-slave-2lzq krb5kdc[15938](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 11:03:21 dist-test-slave-2lzq krb5kdc[15938](info): set up 2 sockets
Sep 29 11:03:21 dist-test-slave-2lzq krb5kdc[15938](info): commencing operation
krb5kdc: starting...
W20260929 11:03:22.678381 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 11:03:22 dist-test-slave-2lzq krb5kdc[15938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T11:03:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T11:03:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 11:03:22.798444 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44819
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44819
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:03:22.877295 15951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:22.877538 15951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:22.877578 15951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:22.878773 15951 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:03:22.878850 15951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:22.878875 15951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:22.878896 15951 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:03:22.878926 15951 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:03:22.880376 15951 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44819
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44819
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:22.880650 15951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:22.880882 15951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:22.885824 15956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:22.885834 15957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:22.885864 15951 server_base.cc:1034] running on GCE node
W20260929 11:03:22.885833 15959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:22.886255 15951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:22.886571 15951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:22.887768 15951 hybrid_clock.cc:648] HybridClock initialized: now 1790679802887751 us; error 46 us; skew 500 ppm
Sep 29 11:03:22 dist-test-slave-2lzq krb5kdc[15938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679802, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:03:22.890226 15951 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 11:03:22.890707 15951 webserver.cc:460] Webserver started at http://127.0.0.1:35503/ using document root <none> and password file <none>
I20260929 11:03:22.890939 15951 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:22.891007 15951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:22.891142 15951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:22.892179 15951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "f1ffb5b72e2848ae9661b56b67fdbe4c"
format_stamp: "Formatted at 2026-09-29 11:03:22 on dist-test-slave-2lzq"
server_key: "d90da4d731341e6ebc9d72fb1d7ed403"
server_key_iv: "490f55c16385883ad371466ae2106863"
server_key_version: "encryptionkey@0"
I20260929 11:03:22.892541 15951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "f1ffb5b72e2848ae9661b56b67fdbe4c"
format_stamp: "Formatted at 2026-09-29 11:03:22 on dist-test-slave-2lzq"
server_key: "d90da4d731341e6ebc9d72fb1d7ed403"
server_key_iv: "490f55c16385883ad371466ae2106863"
server_key_version: "encryptionkey@0"
I20260929 11:03:22.893869 15951 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 11:03:22.894712 15966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:22.894899 15951 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 11:03:22.894997 15951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "f1ffb5b72e2848ae9661b56b67fdbe4c"
format_stamp: "Formatted at 2026-09-29 11:03:22 on dist-test-slave-2lzq"
server_key: "d90da4d731341e6ebc9d72fb1d7ed403"
server_key_iv: "490f55c16385883ad371466ae2106863"
server_key_version: "encryptionkey@0"
I20260929 11:03:22.895084 15951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:22.903910 15951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:22.905645 15951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:22.905803 15951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:22.909797 15951 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44819
I20260929 11:03:22.909830 16018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44819 every 8 connection(s)
I20260929 11:03:22.910220 15951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:03:22.911096 16019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:22.912971 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 15951
I20260929 11:03:22.913081 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 11:03:22.913270 27107 external_mini_cluster.cc:1428] Setting key f3278efd1b1e344496b758d13754fe29
I20260929 11:03:22.913702 16019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1ffb5b72e2848ae9661b56b67fdbe4c: Bootstrap starting.
I20260929 11:03:22.914424 16019 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1ffb5b72e2848ae9661b56b67fdbe4c: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:22.914785 16019 log.cc:824] T 00000000000000000000000000000000 P f1ffb5b72e2848ae9661b56b67fdbe4c: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:22.915498 16019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1ffb5b72e2848ae9661b56b67fdbe4c: No bootstrap required, opened a new log
I20260929 11:03:22.917842 16019 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1ffb5b72e2848ae9661b56b67fdbe4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1ffb5b72e2848ae9661b56b67fdbe4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44819 } }
I20260929 11:03:22.918020 16019 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1ffb5b72e2848ae9661b56b67fdbe4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:22.918095 16019 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1ffb5b72e2848ae9661b56b67fdbe4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1ffb5b72e2848ae9661b56b67fdbe4c, State: Initialized, Role: FOLLOWER
I20260929 11:03:22.918258 16019 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1ffb5b72e2848ae9661b56b67fdbe4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1ffb5b72e2848ae9661b56b67fdbe4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44819 } }
I20260929 11:03:22.918375 16019 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1ffb5b72e2848ae9661b56b67fdbe4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 11:03:22.918419 16019 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1ffb5b72e2848ae9661b56b67fdbe4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 11:03:22.918470 16019 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1ffb5b72e2848ae9661b56b67fdbe4c [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:22.919003 16022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:22.913788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.168:36273 (local address 127.0.0.1:44819)
0929 11:03:22.913933 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:22.913943 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:22.913977 (+ 34us) server_negotiation.cc:407] Connection header received
0929 11:03:22.914454 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:22.914460 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:22.914532 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:22.914672 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:22.914945 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:22.915904 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:22.917016 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:22.917432 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:22.917510 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:22.918270 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:22.918274 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:22.918274 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:22.918493 (+ 219us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:22.918532 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:22.918546 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:22.918623 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:22.918766 (+ 143us) server_negotiation.cc:299] Negotiation successful
0929 11:03:22.918798 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":64,"threads_started":1}
I20260929 11:03:22.919188 16019 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1ffb5b72e2848ae9661b56b67fdbe4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1ffb5b72e2848ae9661b56b67fdbe4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44819 } }
I20260929 11:03:22.919338 16019 leader_election.cc:302] T 00000000000000000000000000000000 P f1ffb5b72e2848ae9661b56b67fdbe4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1ffb5b72e2848ae9661b56b67fdbe4c; no voters:
I20260929 11:03:22.919512 16019 leader_election.cc:288] T 00000000000000000000000000000000 P f1ffb5b72e2848ae9661b56b67fdbe4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 11:03:22.919536 16024 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1ffb5b72e2848ae9661b56b67fdbe4c [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:03:22.919696 16024 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1ffb5b72e2848ae9661b56b67fdbe4c [term 1 LEADER]: Becoming Leader. State: Replica: f1ffb5b72e2848ae9661b56b67fdbe4c, State: Running, Role: LEADER
I20260929 11:03:22.919817 16019 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1ffb5b72e2848ae9661b56b67fdbe4c [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:03:22.919885 16024 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1ffb5b72e2848ae9661b56b67fdbe4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1ffb5b72e2848ae9661b56b67fdbe4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44819 } }
I20260929 11:03:22.919972 27107 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 11:03:22.920280 16026 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1ffb5b72e2848ae9661b56b67fdbe4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1ffb5b72e2848ae9661b56b67fdbe4c. Latest consensus state: current_term: 1 leader_uuid: "f1ffb5b72e2848ae9661b56b67fdbe4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1ffb5b72e2848ae9661b56b67fdbe4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44819 } } }
I20260929 11:03:22.920360 16026 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1ffb5b72e2848ae9661b56b67fdbe4c [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:22.920501 16025 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1ffb5b72e2848ae9661b56b67fdbe4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1ffb5b72e2848ae9661b56b67fdbe4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1ffb5b72e2848ae9661b56b67fdbe4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44819 } } }
I20260929 11:03:22.920553 16025 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1ffb5b72e2848ae9661b56b67fdbe4c [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:22.920795 16031 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:03:22.921288 16031 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:03:22.922989 16031 catalog_manager.cc:1347] Generated new cluster ID: f42c4ee1f2594a8bb1c5e3454a9c3059
I20260929 11:03:22.923055 16031 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 11:03:22.924906 16022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:22.921551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.168:43643 (local address 127.0.0.1:44819)
0929 11:03:22.921587 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:22.921588 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:22.921872 (+ 284us) server_negotiation.cc:407] Connection header received
0929 11:03:22.921917 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:22.921918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:22.921929 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:22.921989 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:22.922151 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:22.922664 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:22.923615 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:22.923925 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:22.923989 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:22.924665 (+ 676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:22.924666 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:22.924667 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:22.924670 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:22.924684 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:22.924688 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:22.924741 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:22.924832 (+ 91us) server_negotiation.cc:299] Negotiation successful
0929 11:03:22.924840 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 11:03:22.931484 16031 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 11:03:22.932163 16031 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:03:22.937783 16031 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1ffb5b72e2848ae9661b56b67fdbe4c: Generated new TSK 0
I20260929 11:03:22.937950 16031 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 11:03:22.943991 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 15951
W20260929 11:03:22.948704 15936 connection.cc:535] client connection to 127.0.0.1:44819 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44819): Connection reset by peer (error 104)
2026-09-29T11:03:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1309 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 11:03:22 dist-test-slave-2lzq krb5kdc[16050](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 11:03:22 dist-test-slave-2lzq krb5kdc[16050](info): set up 2 sockets
Sep 29 11:03:22 dist-test-slave-2lzq krb5kdc[16050](info): commencing operation
krb5kdc: starting...
W20260929 11:03:23.974370 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 11:03:24 dist-test-slave-2lzq krb5kdc[16050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679804, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T11:03:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T11:03:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 11:03:24.092970 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43563
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43563
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:03:24.175586 16063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:24.175812 16063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:24.175836 16063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:24.177012 16063 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:03:24.177049 16063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:24.177063 16063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:24.177076 16063 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:03:24.177088 16063 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:03:24.178573 16063 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43563
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790679549986039-27107-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:24.178774 16063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:24.178947 16063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:24.183944 16068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:24.184039 16069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:24.184144 16063 server_base.cc:1034] running on GCE node
W20260929 11:03:24.183914 16071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:24.184419 16063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:24.184719 16063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:24.185887 16063 hybrid_clock.cc:648] HybridClock initialized: now 1790679804185870 us; error 34 us; skew 500 ppm
Sep 29 11:03:24 dist-test-slave-2lzq krb5kdc[16050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679804, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:03:24.188397 16063 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 11:03:24.188808 16063 webserver.cc:460] Webserver started at http://127.0.0.1:41111/ using document root <none> and password file <none>
I20260929 11:03:24.189034 16063 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:24.189100 16063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:24.189246 16063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:24.190362 16063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "63dc07c0350b42e4b340353a2ac02393"
format_stamp: "Formatted at 2026-09-29 11:03:24 on dist-test-slave-2lzq"
server_key: "646153927272b762f169e840f7cfb654"
server_key_iv: "23dc880d7c5edfc147aaa7da4054e292"
server_key_version: "encryptionkey@0"
I20260929 11:03:24.190708 16063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "63dc07c0350b42e4b340353a2ac02393"
format_stamp: "Formatted at 2026-09-29 11:03:24 on dist-test-slave-2lzq"
server_key: "646153927272b762f169e840f7cfb654"
server_key_iv: "23dc880d7c5edfc147aaa7da4054e292"
server_key_version: "encryptionkey@0"
I20260929 11:03:24.192020 16063 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 11:03:24.192766 16078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:24.192941 16063 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 11:03:24.193044 16063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "63dc07c0350b42e4b340353a2ac02393"
format_stamp: "Formatted at 2026-09-29 11:03:24 on dist-test-slave-2lzq"
server_key: "646153927272b762f169e840f7cfb654"
server_key_iv: "23dc880d7c5edfc147aaa7da4054e292"
server_key_version: "encryptionkey@0"
I20260929 11:03:24.193116 16063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:24.196376 16063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:24.198601 16063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:24.198756 16063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:24.202520 16063 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43563
I20260929 11:03:24.202550 16130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43563 every 8 connection(s)
I20260929 11:03:24.202885 16063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:03:24.203806 16131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:24.206197 16131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63dc07c0350b42e4b340353a2ac02393: Bootstrap starting.
I20260929 11:03:24.206786 16131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63dc07c0350b42e4b340353a2ac02393: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:24.207075 16131 log.cc:824] T 00000000000000000000000000000000 P 63dc07c0350b42e4b340353a2ac02393: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:24.207811 16131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63dc07c0350b42e4b340353a2ac02393: No bootstrap required, opened a new log
I20260929 11:03:24.207854 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 16063
I20260929 11:03:24.207976 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 11:03:24.208155 27107 external_mini_cluster.cc:1428] Setting key 4e4b79b858589d48db43c26adde59c7e
I20260929 11:03:24.209743 16131 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63dc07c0350b42e4b340353a2ac02393 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63dc07c0350b42e4b340353a2ac02393" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43563 } }
I20260929 11:03:24.209910 16131 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63dc07c0350b42e4b340353a2ac02393 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:24.209995 16131 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63dc07c0350b42e4b340353a2ac02393 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63dc07c0350b42e4b340353a2ac02393, State: Initialized, Role: FOLLOWER
I20260929 11:03:24.210242 16131 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63dc07c0350b42e4b340353a2ac02393 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63dc07c0350b42e4b340353a2ac02393" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43563 } }
I20260929 11:03:24.210366 16131 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63dc07c0350b42e4b340353a2ac02393 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 11:03:24.210420 16131 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63dc07c0350b42e4b340353a2ac02393 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 11:03:24.210471 16131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63dc07c0350b42e4b340353a2ac02393 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:24.211014 16135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:24.208687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.168:36519 (local address 127.0.0.1:43563)
0929 11:03:24.208872 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:24.208881 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:24.208912 (+ 31us) server_negotiation.cc:407] Connection header received
0929 11:03:24.209742 (+ 830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:24.209749 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:24.209775 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:24.209920 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:24.210598 (+ 678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:24.210601 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:24.210601 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:24.210605 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:24.210641 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:24.210661 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:24.210732 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:24.210800 (+ 68us) server_negotiation.cc:299] Negotiation successful
0929 11:03:24.210844 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":81,"threads_started":1}
I20260929 11:03:24.211598 16131 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63dc07c0350b42e4b340353a2ac02393 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63dc07c0350b42e4b340353a2ac02393" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43563 } }
I20260929 11:03:24.211735 16131 leader_election.cc:302] T 00000000000000000000000000000000 P 63dc07c0350b42e4b340353a2ac02393 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63dc07c0350b42e4b340353a2ac02393; no voters:
I20260929 11:03:24.211938 16131 leader_election.cc:288] T 00000000000000000000000000000000 P 63dc07c0350b42e4b340353a2ac02393 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 11:03:24.211953 27107 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 11:03:24.211961 16136 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63dc07c0350b42e4b340353a2ac02393 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:03:24.212076 16136 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63dc07c0350b42e4b340353a2ac02393 [term 1 LEADER]: Becoming Leader. State: Replica: 63dc07c0350b42e4b340353a2ac02393, State: Running, Role: LEADER
I20260929 11:03:24.212203 16131 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63dc07c0350b42e4b340353a2ac02393 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:03:24.212209 16136 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63dc07c0350b42e4b340353a2ac02393 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63dc07c0350b42e4b340353a2ac02393" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43563 } }
I20260929 11:03:24.212641 16137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63dc07c0350b42e4b340353a2ac02393 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63dc07c0350b42e4b340353a2ac02393" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63dc07c0350b42e4b340353a2ac02393" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43563 } } }
I20260929 11:03:24.212652 16138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63dc07c0350b42e4b340353a2ac02393 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63dc07c0350b42e4b340353a2ac02393. Latest consensus state: current_term: 1 leader_uuid: "63dc07c0350b42e4b340353a2ac02393" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63dc07c0350b42e4b340353a2ac02393" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43563 } } }
I20260929 11:03:24.212808 16137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63dc07c0350b42e4b340353a2ac02393 [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:24.212822 16138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63dc07c0350b42e4b340353a2ac02393 [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:24.213725 16144 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:03:24.214385 16144 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:03:24.214819 16135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:24.213656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.168:34997 (local address 127.0.0.1:43563)
0929 11:03:24.213689 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:24.213691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:24.213734 (+ 43us) server_negotiation.cc:407] Connection header received
0929 11:03:24.213883 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:24.213884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:24.213889 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:24.213920 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:24.214571 (+ 651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:24.214573 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:24.214573 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:24.214578 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:24.214593 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:24.214598 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:24.214654 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:24.214749 (+ 95us) server_negotiation.cc:299] Negotiation successful
0929 11:03:24.214763 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260929 11:03:24.215924 16144 catalog_manager.cc:1347] Generated new cluster ID: e80720fbaee34ed68082a161bdbcb891
I20260929 11:03:24.215974 16144 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 11:03:24.242010 16144 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 11:03:24.242269 16144 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:03:24.248594 16144 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63dc07c0350b42e4b340353a2ac02393: Generated new TSK 0
I20260929 11:03:24.248795 16144 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 11:03:24.255960 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 16063
2026-09-29T11:03:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1311 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 11:03:24 dist-test-slave-2lzq krb5kdc[16162](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 11:03:24 dist-test-slave-2lzq krb5kdc[16162](info): set up 2 sockets
Sep 29 11:03:24 dist-test-slave-2lzq krb5kdc[16162](info): commencing operation
krb5kdc: starting...
W20260929 11:03:25.290540 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 11:03:25 dist-test-slave-2lzq krb5kdc[16162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T11:03:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T11:03:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.120.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
I20260929 11:03:25.409408 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:39041
--webserver_interface=127.26.120.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:39115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:39041
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:03:25.489711 16175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:25.489970 16175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:25.490010 16175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:25.491180 16175 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:03:25.491264 16175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:25.491290 16175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:25.491320 16175 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:03:25.491343 16175 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:03:25.492816 16175 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:39115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:39041
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:39041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:25.493094 16175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:25.493327 16175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:25.498320 16180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:25.498353 16175 server_base.cc:1034] running on GCE node
W20260929 11:03:25.498363 16183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:25.498324 16181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:25.498752 16175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:25.499054 16175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:25.500213 16175 hybrid_clock.cc:648] HybridClock initialized: now 1790679805500200 us; error 29 us; skew 500 ppm
Sep 29 11:03:25 dist-test-slave-2lzq krb5kdc[16162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679805, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:03:25.506899 16175 init.cc:377] Logged in from keytab as kudu/127.26.120.254@KRBTEST.COM (short username kudu)
I20260929 11:03:25.507356 16175 webserver.cc:460] Webserver started at http://127.26.120.254:41561/ using document root <none> and password file <none>
I20260929 11:03:25.507560 16175 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:25.507601 16175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:25.507688 16175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:25.508671 16175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "386f73416f1947a9bab1cb569a858f3f"
format_stamp: "Formatted at 2026-09-29 11:03:25 on dist-test-slave-2lzq"
server_key: "448232065059d77ecbc04c36b8a1214f"
server_key_iv: "55e4c36b9804ea69eaa0c675acba9da2"
server_key_version: "encryptionkey@0"
I20260929 11:03:25.508955 16175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "386f73416f1947a9bab1cb569a858f3f"
format_stamp: "Formatted at 2026-09-29 11:03:25 on dist-test-slave-2lzq"
server_key: "448232065059d77ecbc04c36b8a1214f"
server_key_iv: "55e4c36b9804ea69eaa0c675acba9da2"
server_key_version: "encryptionkey@0"
I20260929 11:03:25.510239 16175 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 11:03:25.510923 16190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:25.511121 16175 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 11:03:25.511204 16175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "386f73416f1947a9bab1cb569a858f3f"
format_stamp: "Formatted at 2026-09-29 11:03:25 on dist-test-slave-2lzq"
server_key: "448232065059d77ecbc04c36b8a1214f"
server_key_iv: "55e4c36b9804ea69eaa0c675acba9da2"
server_key_version: "encryptionkey@0"
I20260929 11:03:25.511257 16175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:25.528093 16175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:25.530102 16175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:25.530238 16175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:25.534026 16175 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:39041
I20260929 11:03:25.534133 16242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:39041 every 8 connection(s)
I20260929 11:03:25.534485 16175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:03:25.535380 16243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:25.537554 16243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 386f73416f1947a9bab1cb569a858f3f: Bootstrap starting.
I20260929 11:03:25.538136 16243 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 386f73416f1947a9bab1cb569a858f3f: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:25.538388 16243 log.cc:824] T 00000000000000000000000000000000 P 386f73416f1947a9bab1cb569a858f3f: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:25.539096 16243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 386f73416f1947a9bab1cb569a858f3f: No bootstrap required, opened a new log
I20260929 11:03:25.540606 16243 raft_consensus.cc:348] T 00000000000000000000000000000000 P 386f73416f1947a9bab1cb569a858f3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "386f73416f1947a9bab1cb569a858f3f" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 39041 } }
I20260929 11:03:25.540741 16243 raft_consensus.cc:374] T 00000000000000000000000000000000 P 386f73416f1947a9bab1cb569a858f3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:25.540763 16243 raft_consensus.cc:729] T 00000000000000000000000000000000 P 386f73416f1947a9bab1cb569a858f3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 386f73416f1947a9bab1cb569a858f3f, State: Initialized, Role: FOLLOWER
I20260929 11:03:25.540884 16243 consensus_queue.cc:260] T 00000000000000000000000000000000 P 386f73416f1947a9bab1cb569a858f3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "386f73416f1947a9bab1cb569a858f3f" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 39041 } }
I20260929 11:03:25.540935 16243 raft_consensus.cc:388] T 00000000000000000000000000000000 P 386f73416f1947a9bab1cb569a858f3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 11:03:25.540956 16243 raft_consensus.cc:482] T 00000000000000000000000000000000 P 386f73416f1947a9bab1cb569a858f3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 11:03:25.540987 16243 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 386f73416f1947a9bab1cb569a858f3f [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:25.541759 16243 raft_consensus.cc:504] T 00000000000000000000000000000000 P 386f73416f1947a9bab1cb569a858f3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "386f73416f1947a9bab1cb569a858f3f" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 39041 } }
I20260929 11:03:25.541851 16243 leader_election.cc:302] T 00000000000000000000000000000000 P 386f73416f1947a9bab1cb569a858f3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 386f73416f1947a9bab1cb569a858f3f; no voters:
I20260929 11:03:25.541986 16243 leader_election.cc:288] T 00000000000000000000000000000000 P 386f73416f1947a9bab1cb569a858f3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 11:03:25.542049 16246 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 386f73416f1947a9bab1cb569a858f3f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:03:25.542212 16246 raft_consensus.cc:686] T 00000000000000000000000000000000 P 386f73416f1947a9bab1cb569a858f3f [term 1 LEADER]: Becoming Leader. State: Replica: 386f73416f1947a9bab1cb569a858f3f, State: Running, Role: LEADER
I20260929 11:03:25.542277 16243 sys_catalog.cc:564] T 00000000000000000000000000000000 P 386f73416f1947a9bab1cb569a858f3f [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:03:25.542300 16246 consensus_queue.cc:237] T 00000000000000000000000000000000 P 386f73416f1947a9bab1cb569a858f3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "386f73416f1947a9bab1cb569a858f3f" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 39041 } }
I20260929 11:03:25.542704 16247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 386f73416f1947a9bab1cb569a858f3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "386f73416f1947a9bab1cb569a858f3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "386f73416f1947a9bab1cb569a858f3f" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 39041 } } }
I20260929 11:03:25.542706 16248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 386f73416f1947a9bab1cb569a858f3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 386f73416f1947a9bab1cb569a858f3f. Latest consensus state: current_term: 1 leader_uuid: "386f73416f1947a9bab1cb569a858f3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "386f73416f1947a9bab1cb569a858f3f" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 39041 } } }
I20260929 11:03:25.542853 16247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 386f73416f1947a9bab1cb569a858f3f [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:25.542862 16248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 386f73416f1947a9bab1cb569a858f3f [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:25.543118 16255 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:03:25.543699 16255 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:03:25.544411 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 16175
I20260929 11:03:25.544515 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 11:03:25.544682 27107 external_mini_cluster.cc:1428] Setting key 6ea8182c7a73fd54e1ea661c928b0b65
I20260929 11:03:25.545495 16255 catalog_manager.cc:1347] Generated new cluster ID: 5261d5c834654ed9a88f0ff696a1a5a9
I20260929 11:03:25.545547 16255 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 11:03:25.552493 16255 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 11:03:25.553042 16255 catalog_manager.cc:1504] Loading token signing keys...
Sep 29 11:03:25 dist-test-slave-2lzq krb5kdc[16162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:03:25.557713 16266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:25.545161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40251 (local address 127.26.120.254:39041)
0929 11:03:25.546168 (+ 1007us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:25.546177 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:25.546211 (+ 34us) server_negotiation.cc:407] Connection header received
0929 11:03:25.546334 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:25.546341 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:25.546412 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:25.546553 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:25.547301 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:25.548761 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:25.549838 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:25.550871 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:25.550983 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:25.553909 (+ 2926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:25.553914 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:25.553914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:25.553926 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:25.555951 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:25.556424 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:25.556427 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:25.556427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:25.556531 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:25.556772 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:25.556773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:25.556774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:25.557046 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:25.557194 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:25.557366 (+ 172us) server_negotiation.cc:299] Negotiation successful
0929 11:03:25.557474 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":962,"thread_start_us":72,"threads_started":1}
I20260929 11:03:25.558884 27107 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 11:03:25.560632 16255 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 386f73416f1947a9bab1cb569a858f3f: Generated new TSK 0
I20260929 11:03:25.560811 16255 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 11:03:25.569408 16266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:25.560437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50925 (local address 127.26.120.254:39041)
0929 11:03:25.560472 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:25.560474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:25.562330 (+ 1856us) server_negotiation.cc:407] Connection header received
0929 11:03:25.562395 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:25.562396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:25.562405 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:25.562459 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:25.563136 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:25.563654 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:25.564748 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:25.565209 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:25.565266 (+ 57us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:25.566734 (+ 1468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:25.566735 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:25.566735 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:25.566742 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:25.568138 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:25.568559 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:25.568560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:25.568560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:25.568614 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:25.568898 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:25.568899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:25.568899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:25.568998 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:25.569073 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:25.569239 (+ 166us) server_negotiation.cc:299] Negotiation successful
0929 11:03:25.569291 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260929 11:03:25.571833 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 16175
W20260929 11:03:25.575974 16160 connection.cc:535] client connection to 127.26.120.254:39041 recv error: Network error: failed to read from TLS socket (remote: 127.26.120.254:39041): Connection reset by peer (error 104)
2026-09-29T11:03:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1315 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 11:03:25 dist-test-slave-2lzq krb5kdc[16274](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 11:03:25 dist-test-slave-2lzq krb5kdc[16274](info): set up 2 sockets
Sep 29 11:03:25 dist-test-slave-2lzq krb5kdc[16274](info): commencing operation
krb5kdc: starting...
W20260929 11:03:26.606376 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 11:03:26 dist-test-slave-2lzq krb5kdc[16274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T11:03:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T11:03:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.120.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
I20260929 11:03:26.726848 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:40389
--webserver_interface=127.26.120.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:44521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:40389
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:03:26.806439 16287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:26.806635 16287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:26.806658 16287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:26.807706 16287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:03:26.807744 16287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:26.807757 16287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:26.807770 16287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:03:26.807781 16287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:03:26.809135 16287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:44521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:40389
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:40389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:26.809316 16287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:26.809480 16287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:26.814338 16293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:26.814445 16295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:26.814311 16292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:26.814323 16287 server_base.cc:1034] running on GCE node
I20260929 11:03:26.814769 16287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:26.815001 16287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:26.816138 16287 hybrid_clock.cc:648] HybridClock initialized: now 1790679806816127 us; error 27 us; skew 500 ppm
Sep 29 11:03:26 dist-test-slave-2lzq krb5kdc[16274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679806, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:03:26.818600 16287 init.cc:377] Logged in from keytab as kudu/127.26.120.254@KRBTEST.COM (short username kudu)
I20260929 11:03:26.819061 16287 webserver.cc:460] Webserver started at http://127.26.120.254:40479/ using document root <none> and password file <none>
I20260929 11:03:26.819293 16287 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:26.819339 16287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:26.819464 16287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:26.820462 16287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "12858f45abab4c5a8943386eaf8ccbc6"
format_stamp: "Formatted at 2026-09-29 11:03:26 on dist-test-slave-2lzq"
server_key: "8e75ef859e0d5ee5ee63a8364f6a6c83"
server_key_iv: "9e02c6116ce146a0cd1116edb1e77894"
server_key_version: "encryptionkey@0"
I20260929 11:03:26.820806 16287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "12858f45abab4c5a8943386eaf8ccbc6"
format_stamp: "Formatted at 2026-09-29 11:03:26 on dist-test-slave-2lzq"
server_key: "8e75ef859e0d5ee5ee63a8364f6a6c83"
server_key_iv: "9e02c6116ce146a0cd1116edb1e77894"
server_key_version: "encryptionkey@0"
I20260929 11:03:26.822167 16287 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 11:03:26.822906 16302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:26.823081 16287 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 11:03:26.823168 16287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "12858f45abab4c5a8943386eaf8ccbc6"
format_stamp: "Formatted at 2026-09-29 11:03:26 on dist-test-slave-2lzq"
server_key: "8e75ef859e0d5ee5ee63a8364f6a6c83"
server_key_iv: "9e02c6116ce146a0cd1116edb1e77894"
server_key_version: "encryptionkey@0"
I20260929 11:03:26.823230 16287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:26.852824 16287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:26.855204 16287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:26.855316 16287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:26.859051 16287 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:40389
I20260929 11:03:26.859102 16354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:40389 every 8 connection(s)
I20260929 11:03:26.859458 16287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:03:26.860364 16355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:26.861644 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 16287
I20260929 11:03:26.861771 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 11:03:26.861964 27107 external_mini_cluster.cc:1428] Setting key a45fc5afb42774cfc449821c654046a9
I20260929 11:03:26.863052 16355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12858f45abab4c5a8943386eaf8ccbc6: Bootstrap starting.
I20260929 11:03:26.863751 16355 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12858f45abab4c5a8943386eaf8ccbc6: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:26.864041 16355 log.cc:824] T 00000000000000000000000000000000 P 12858f45abab4c5a8943386eaf8ccbc6: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:26.864748 16355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12858f45abab4c5a8943386eaf8ccbc6: No bootstrap required, opened a new log
I20260929 11:03:26.867172 16355 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12858f45abab4c5a8943386eaf8ccbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12858f45abab4c5a8943386eaf8ccbc6" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 40389 } }
I20260929 11:03:26.867301 16355 raft_consensus.cc:374] T 00000000000000000000000000000000 P 12858f45abab4c5a8943386eaf8ccbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:26.867323 16355 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12858f45abab4c5a8943386eaf8ccbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12858f45abab4c5a8943386eaf8ccbc6, State: Initialized, Role: FOLLOWER
I20260929 11:03:26.867460 16355 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12858f45abab4c5a8943386eaf8ccbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12858f45abab4c5a8943386eaf8ccbc6" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 40389 } }
I20260929 11:03:26.867552 16355 raft_consensus.cc:388] T 00000000000000000000000000000000 P 12858f45abab4c5a8943386eaf8ccbc6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 11:03:26.867609 16355 raft_consensus.cc:482] T 00000000000000000000000000000000 P 12858f45abab4c5a8943386eaf8ccbc6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 11:03:26.867664 16355 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12858f45abab4c5a8943386eaf8ccbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:26.867887 16358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:26.862514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39221 (local address 127.26.120.254:40389)
0929 11:03:26.862691 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:26.862701 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:26.862733 (+ 32us) server_negotiation.cc:407] Connection header received
0929 11:03:26.863130 (+ 397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:26.863136 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:26.863219 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:26.863350 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:26.863619 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:26.864736 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:26.865839 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:26.866313 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:26.866397 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:26.867121 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:26.867125 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:26.867125 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:26.867282 (+ 157us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:26.867341 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:26.867358 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:26.867469 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:26.867553 (+ 84us) server_negotiation.cc:299] Negotiation successful
0929 11:03:26.867697 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":84,"threads_started":1}
I20260929 11:03:26.868309 16355 raft_consensus.cc:504] T 00000000000000000000000000000000 P 12858f45abab4c5a8943386eaf8ccbc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12858f45abab4c5a8943386eaf8ccbc6" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 40389 } }
I20260929 11:03:26.868438 16355 leader_election.cc:302] T 00000000000000000000000000000000 P 12858f45abab4c5a8943386eaf8ccbc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 12858f45abab4c5a8943386eaf8ccbc6; no voters:
I20260929 11:03:26.868602 16355 leader_election.cc:288] T 00000000000000000000000000000000 P 12858f45abab4c5a8943386eaf8ccbc6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 11:03:26.868624 16360 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 12858f45abab4c5a8943386eaf8ccbc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:03:26.868791 27107 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 11:03:26.868817 16360 raft_consensus.cc:686] T 00000000000000000000000000000000 P 12858f45abab4c5a8943386eaf8ccbc6 [term 1 LEADER]: Becoming Leader. State: Replica: 12858f45abab4c5a8943386eaf8ccbc6, State: Running, Role: LEADER
I20260929 11:03:26.868881 16355 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12858f45abab4c5a8943386eaf8ccbc6 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:03:26.868901 16360 consensus_queue.cc:237] T 00000000000000000000000000000000 P 12858f45abab4c5a8943386eaf8ccbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12858f45abab4c5a8943386eaf8ccbc6" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 40389 } }
I20260929 11:03:26.869204 16360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12858f45abab4c5a8943386eaf8ccbc6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12858f45abab4c5a8943386eaf8ccbc6. Latest consensus state: current_term: 1 leader_uuid: "12858f45abab4c5a8943386eaf8ccbc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12858f45abab4c5a8943386eaf8ccbc6" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 40389 } } }
I20260929 11:03:26.869220 16362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12858f45abab4c5a8943386eaf8ccbc6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "12858f45abab4c5a8943386eaf8ccbc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12858f45abab4c5a8943386eaf8ccbc6" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 40389 } } }
I20260929 11:03:26.869293 16360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12858f45abab4c5a8943386eaf8ccbc6 [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:26.869343 16362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12858f45abab4c5a8943386eaf8ccbc6 [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:26.869745 16366 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:03:26.870306 16366 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:03:26.871851 16366 catalog_manager.cc:1347] Generated new cluster ID: 271d38d137734ffdae72df3a39afe696
I20260929 11:03:26.871898 16366 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 11:03:26.873534 16358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:26.870364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42231 (local address 127.26.120.254:40389)
0929 11:03:26.870401 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:26.870403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:26.870526 (+ 123us) server_negotiation.cc:407] Connection header received
0929 11:03:26.870626 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:26.870627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:26.870636 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:26.870687 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:26.870929 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:26.871492 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:26.872339 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:26.872559 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:26.872643 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:26.873296 (+ 653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:26.873298 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:26.873298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:26.873302 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:26.873315 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:26.873318 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:26.873372 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:26.873434 (+ 62us) server_negotiation.cc:299] Negotiation successful
0929 11:03:26.873442 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 11:03:26.879166 16366 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 11:03:26.879657 16366 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:03:26.887986 16366 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 12858f45abab4c5a8943386eaf8ccbc6: Generated new TSK 0
I20260929 11:03:26.888170 16366 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 11:03:26.893393 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 16287
W20260929 11:03:26.897677 16272 connection.cc:535] client connection to 127.26.120.254:40389 recv error: Network error: failed to read from TLS socket (remote: 127.26.120.254:40389): Connection reset by peer (error 104)
2026-09-29T11:03:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1321 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 11:03:26 dist-test-slave-2lzq krb5kdc[16386](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 11:03:26 dist-test-slave-2lzq krb5kdc[16386](info): set up 2 sockets
Sep 29 11:03:26 dist-test-slave-2lzq krb5kdc[16386](info): commencing operation
krb5kdc: starting...
W20260929 11:03:27.938436 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 11:03:28 dist-test-slave-2lzq krb5kdc[16386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679808, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T11:03:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T11:03:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.120.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
I20260929 11:03:28.054405 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:41051
--webserver_interface=127.26.120.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:34615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:41051
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:03:28.135637 16399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:28.135888 16399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:28.135926 16399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:28.137076 16399 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:03:28.137166 16399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:28.137193 16399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:28.137225 16399 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:03:28.137249 16399 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:03:28.138834 16399 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:34615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:41051
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:41051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:28.139118 16399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:28.139355 16399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 11:03:28.144260 16399 server_base.cc:1034] running on GCE node
W20260929 11:03:28.144230 16405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:28.144290 16407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:28.144222 16404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:28.144724 16399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:28.145041 16399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:28.146209 16399 hybrid_clock.cc:648] HybridClock initialized: now 1790679808146206 us; error 39 us; skew 500 ppm
Sep 29 11:03:28 dist-test-slave-2lzq krb5kdc[16386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679808, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:03:28.148691 16399 init.cc:377] Logged in from keytab as kudu/127.26.120.254@KRBTEST.COM (short username kudu)
I20260929 11:03:28.149119 16399 webserver.cc:460] Webserver started at http://127.26.120.254:32967/ using document root <none> and password file <none>
I20260929 11:03:28.149359 16399 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:28.149433 16399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:28.149580 16399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:28.150682 16399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "58e8c762089946759a9cfdb059a34c37"
format_stamp: "Formatted at 2026-09-29 11:03:28 on dist-test-slave-2lzq"
server_key: "960eed3f36371d78ce438f4c42ab8c33"
server_key_iv: "4bb138a528723d3a002e8391a25145f6"
server_key_version: "encryptionkey@0"
I20260929 11:03:28.151048 16399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "58e8c762089946759a9cfdb059a34c37"
format_stamp: "Formatted at 2026-09-29 11:03:28 on dist-test-slave-2lzq"
server_key: "960eed3f36371d78ce438f4c42ab8c33"
server_key_iv: "4bb138a528723d3a002e8391a25145f6"
server_key_version: "encryptionkey@0"
I20260929 11:03:28.152539 16399 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 11:03:28.153244 16414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:28.153440 16399 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 11:03:28.153537 16399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "58e8c762089946759a9cfdb059a34c37"
format_stamp: "Formatted at 2026-09-29 11:03:28 on dist-test-slave-2lzq"
server_key: "960eed3f36371d78ce438f4c42ab8c33"
server_key_iv: "4bb138a528723d3a002e8391a25145f6"
server_key_version: "encryptionkey@0"
I20260929 11:03:28.153621 16399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:28.157060 16399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:28.158947 16399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:28.159099 16399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:28.163015 16399 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:41051
I20260929 11:03:28.163040 16466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:41051 every 8 connection(s)
I20260929 11:03:28.163367 16399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:03:28.164444 16467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:28.166666 16467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58e8c762089946759a9cfdb059a34c37: Bootstrap starting.
I20260929 11:03:28.167218 16467 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58e8c762089946759a9cfdb059a34c37: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:28.167496 16467 log.cc:824] T 00000000000000000000000000000000 P 58e8c762089946759a9cfdb059a34c37: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:28.168139 16467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58e8c762089946759a9cfdb059a34c37: No bootstrap required, opened a new log
I20260929 11:03:28.169036 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 16399
I20260929 11:03:28.169166 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 11:03:28.169346 27107 external_mini_cluster.cc:1428] Setting key bc24c7151c1d3752e469a5666881a619
I20260929 11:03:28.169984 16467 raft_consensus.cc:348] T 00000000000000000000000000000000 P 58e8c762089946759a9cfdb059a34c37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58e8c762089946759a9cfdb059a34c37" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 41051 } }
I20260929 11:03:28.170199 16467 raft_consensus.cc:374] T 00000000000000000000000000000000 P 58e8c762089946759a9cfdb059a34c37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:28.170249 16467 raft_consensus.cc:729] T 00000000000000000000000000000000 P 58e8c762089946759a9cfdb059a34c37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58e8c762089946759a9cfdb059a34c37, State: Initialized, Role: FOLLOWER
I20260929 11:03:28.170377 16467 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58e8c762089946759a9cfdb059a34c37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58e8c762089946759a9cfdb059a34c37" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 41051 } }
I20260929 11:03:28.170470 16467 raft_consensus.cc:388] T 00000000000000000000000000000000 P 58e8c762089946759a9cfdb059a34c37 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 11:03:28.170527 16467 raft_consensus.cc:482] T 00000000000000000000000000000000 P 58e8c762089946759a9cfdb059a34c37 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 11:03:28.170580 16467 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 58e8c762089946759a9cfdb059a34c37 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:28.171425 16471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:28.169864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55477 (local address 127.26.120.254:41051)
0929 11:03:28.170084 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:28.170093 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:28.170124 (+ 31us) server_negotiation.cc:407] Connection header received
0929 11:03:28.170244 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:28.170252 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:28.170282 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:28.170420 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:28.171006 (+ 586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:28.171009 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:28.171010 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:28.171014 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:28.171058 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:28.171069 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:28.171140 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:28.171220 (+ 80us) server_negotiation.cc:299] Negotiation successful
0929 11:03:28.171261 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":98,"threads_started":1}
I20260929 11:03:28.171878 16467 raft_consensus.cc:504] T 00000000000000000000000000000000 P 58e8c762089946759a9cfdb059a34c37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58e8c762089946759a9cfdb059a34c37" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 41051 } }
I20260929 11:03:28.171991 16467 leader_election.cc:302] T 00000000000000000000000000000000 P 58e8c762089946759a9cfdb059a34c37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 58e8c762089946759a9cfdb059a34c37; no voters:
I20260929 11:03:28.172142 16467 leader_election.cc:288] T 00000000000000000000000000000000 P 58e8c762089946759a9cfdb059a34c37 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 11:03:28.172171 16472 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 58e8c762089946759a9cfdb059a34c37 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:03:28.172310 16472 raft_consensus.cc:686] T 00000000000000000000000000000000 P 58e8c762089946759a9cfdb059a34c37 [term 1 LEADER]: Becoming Leader. State: Replica: 58e8c762089946759a9cfdb059a34c37, State: Running, Role: LEADER
I20260929 11:03:28.172334 27107 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 11:03:28.172387 16467 sys_catalog.cc:564] T 00000000000000000000000000000000 P 58e8c762089946759a9cfdb059a34c37 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:03:28.172379 16472 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58e8c762089946759a9cfdb059a34c37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58e8c762089946759a9cfdb059a34c37" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 41051 } }
I20260929 11:03:28.172726 16473 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58e8c762089946759a9cfdb059a34c37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "58e8c762089946759a9cfdb059a34c37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58e8c762089946759a9cfdb059a34c37" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 41051 } } }
I20260929 11:03:28.172802 16473 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58e8c762089946759a9cfdb059a34c37 [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:28.172936 16473 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58e8c762089946759a9cfdb059a34c37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58e8c762089946759a9cfdb059a34c37. Latest consensus state: current_term: 1 leader_uuid: "58e8c762089946759a9cfdb059a34c37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58e8c762089946759a9cfdb059a34c37" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 41051 } } }
I20260929 11:03:28.172987 16473 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58e8c762089946759a9cfdb059a34c37 [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:28.173334 16478 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:03:28.173813 16478 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:03:28.175223 16471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:28.173983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59357 (local address 127.26.120.254:41051)
0929 11:03:28.174017 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:28.174019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:28.174233 (+ 214us) server_negotiation.cc:407] Connection header received
0929 11:03:28.174324 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:28.174325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:28.174330 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:28.174357 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:28.174987 (+ 630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:28.174988 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:28.174988 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:28.174993 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:28.175005 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:28.175009 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:28.175066 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:28.175142 (+ 76us) server_negotiation.cc:299] Negotiation successful
0929 11:03:28.175158 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260929 11:03:28.175774 16478 catalog_manager.cc:1347] Generated new cluster ID: 8c7e12b654a24da89bc570d6a6f2d4cd
I20260929 11:03:28.175829 16478 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 11:03:28.191819 16478 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 11:03:28.192025 16478 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:03:28.198417 16478 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 58e8c762089946759a9cfdb059a34c37: Generated new TSK 0
I20260929 11:03:28.198627 16478 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 11:03:28.200769 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 16399
2026-09-29T11:03:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1307 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 11:03:28 dist-test-slave-2lzq krb5kdc[16498](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 11:03:28 dist-test-slave-2lzq krb5kdc[16498](info): set up 2 sockets
Sep 29 11:03:28 dist-test-slave-2lzq krb5kdc[16498](info): commencing operation
krb5kdc: starting...
W20260929 11:03:29.258493 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 11:03:29 dist-test-slave-2lzq krb5kdc[16498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679809, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T11:03:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T11:03:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.120.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
I20260929 11:03:29.375087 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:45335
--webserver_interface=127.26.120.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:36563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:45335
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:03:29.454612 16511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:29.454917 16511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:29.454957 16511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:29.456112 16511 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:03:29.456180 16511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:29.456203 16511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:29.456225 16511 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:03:29.456254 16511 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:03:29.457724 16511 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:36563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:45335
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:45335
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16511
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:29.458002 16511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:29.458241 16511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:29.462945 16516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:29.462980 16517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:29.463011 16519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:29.463128 16511 server_base.cc:1034] running on GCE node
I20260929 11:03:29.463445 16511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:29.463676 16511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:29.464815 16511 hybrid_clock.cc:648] HybridClock initialized: now 1790679809464793 us; error 39 us; skew 500 ppm
Sep 29 11:03:29 dist-test-slave-2lzq krb5kdc[16498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679809, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:03:29.467250 16511 init.cc:377] Logged in from keytab as kudu/127.26.120.254@KRBTEST.COM (short username kudu)
I20260929 11:03:29.467708 16511 webserver.cc:460] Webserver started at http://127.26.120.254:33147/ using document root <none> and password file <none>
I20260929 11:03:29.467952 16511 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:29.468027 16511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:29.468180 16511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:29.469203 16511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "04a8ff76122f40cf87dabc7f44bee1c6"
format_stamp: "Formatted at 2026-09-29 11:03:29 on dist-test-slave-2lzq"
server_key: "39682f7b26b0bd5fa59ec1dd3e9f5764"
server_key_iv: "c817bc81992d172f718a4a28be6d6d28"
server_key_version: "encryptionkey@0"
I20260929 11:03:29.469568 16511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "04a8ff76122f40cf87dabc7f44bee1c6"
format_stamp: "Formatted at 2026-09-29 11:03:29 on dist-test-slave-2lzq"
server_key: "39682f7b26b0bd5fa59ec1dd3e9f5764"
server_key_iv: "c817bc81992d172f718a4a28be6d6d28"
server_key_version: "encryptionkey@0"
I20260929 11:03:29.470973 16511 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 11:03:29.471733 16526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:29.471916 16511 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 11:03:29.472018 16511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "04a8ff76122f40cf87dabc7f44bee1c6"
format_stamp: "Formatted at 2026-09-29 11:03:29 on dist-test-slave-2lzq"
server_key: "39682f7b26b0bd5fa59ec1dd3e9f5764"
server_key_iv: "c817bc81992d172f718a4a28be6d6d28"
server_key_version: "encryptionkey@0"
I20260929 11:03:29.472096 16511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:29.479104 16511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:29.480805 16511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:29.480964 16511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:29.484995 16511 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:45335
I20260929 11:03:29.485025 16578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:45335 every 8 connection(s)
I20260929 11:03:29.485356 16511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:03:29.486224 16579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:29.488366 16579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04a8ff76122f40cf87dabc7f44bee1c6: Bootstrap starting.
I20260929 11:03:29.488871 16579 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04a8ff76122f40cf87dabc7f44bee1c6: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:29.489099 16579 log.cc:824] T 00000000000000000000000000000000 P 04a8ff76122f40cf87dabc7f44bee1c6: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:29.489573 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 16511
I20260929 11:03:29.489687 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 11:03:29.489813 16579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04a8ff76122f40cf87dabc7f44bee1c6: No bootstrap required, opened a new log
I20260929 11:03:29.489928 27107 external_mini_cluster.cc:1428] Setting key 134205510c9a97758fb4ebf714b57d4e
I20260929 11:03:29.492269 16579 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04a8ff76122f40cf87dabc7f44bee1c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04a8ff76122f40cf87dabc7f44bee1c6" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 45335 } }
I20260929 11:03:29.492406 16579 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04a8ff76122f40cf87dabc7f44bee1c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:29.492429 16579 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04a8ff76122f40cf87dabc7f44bee1c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04a8ff76122f40cf87dabc7f44bee1c6, State: Initialized, Role: FOLLOWER
I20260929 11:03:29.492551 16579 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04a8ff76122f40cf87dabc7f44bee1c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04a8ff76122f40cf87dabc7f44bee1c6" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 45335 } }
I20260929 11:03:29.492628 16579 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04a8ff76122f40cf87dabc7f44bee1c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 11:03:29.492650 16579 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04a8ff76122f40cf87dabc7f44bee1c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 11:03:29.492710 16579 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04a8ff76122f40cf87dabc7f44bee1c6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:29.493268 16579 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04a8ff76122f40cf87dabc7f44bee1c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04a8ff76122f40cf87dabc7f44bee1c6" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 45335 } }
I20260929 11:03:29.493359 16579 leader_election.cc:302] T 00000000000000000000000000000000 P 04a8ff76122f40cf87dabc7f44bee1c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04a8ff76122f40cf87dabc7f44bee1c6; no voters:
I20260929 11:03:29.493559 16579 leader_election.cc:288] T 00000000000000000000000000000000 P 04a8ff76122f40cf87dabc7f44bee1c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 11:03:29.493577 16584 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04a8ff76122f40cf87dabc7f44bee1c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:03:29.493780 16584 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04a8ff76122f40cf87dabc7f44bee1c6 [term 1 LEADER]: Becoming Leader. State: Replica: 04a8ff76122f40cf87dabc7f44bee1c6, State: Running, Role: LEADER
I20260929 11:03:29.493867 16579 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04a8ff76122f40cf87dabc7f44bee1c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:03:29.493986 16584 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04a8ff76122f40cf87dabc7f44bee1c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04a8ff76122f40cf87dabc7f44bee1c6" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 45335 } }
I20260929 11:03:29.494328 16584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04a8ff76122f40cf87dabc7f44bee1c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04a8ff76122f40cf87dabc7f44bee1c6. Latest consensus state: current_term: 1 leader_uuid: "04a8ff76122f40cf87dabc7f44bee1c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04a8ff76122f40cf87dabc7f44bee1c6" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 45335 } } }
I20260929 11:03:29.494338 16585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04a8ff76122f40cf87dabc7f44bee1c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04a8ff76122f40cf87dabc7f44bee1c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04a8ff76122f40cf87dabc7f44bee1c6" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 45335 } } }
I20260929 11:03:29.494477 16585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04a8ff76122f40cf87dabc7f44bee1c6 [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:29.494671 16584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04a8ff76122f40cf87dabc7f44bee1c6 [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:29.494832 16590 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:03:29.495329 16590 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:03:29.497007 16590 catalog_manager.cc:1347] Generated new cluster ID: 98853da62f0f4f0297449b6f42907a8e
I20260929 11:03:29.497061 16590 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 11:03:29 dist-test-slave-2lzq krb5kdc[16498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679809, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:03:29.502187 16583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:29.490507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52965 (local address 127.26.120.254:45335)
0929 11:03:29.490676 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:29.490692 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:29.490720 (+ 28us) server_negotiation.cc:407] Connection header received
0929 11:03:29.491206 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:29.491212 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:29.491257 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:29.491378 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:29.492159 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:29.493515 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:29.494690 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:29.495295 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:29.495425 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:29.498610 (+ 3185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:29.498614 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:29.498614 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:29.498625 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:29.500493 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:29.500906 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:29.500910 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:29.500910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:29.501010 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:29.501226 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:29.501226 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:29.501227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:29.501509 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:29.501649 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:29.501847 (+ 198us) server_negotiation.cc:299] Negotiation successful
0929 11:03:29.501976 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260929 11:03:29.503194 27107 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 11:03:29.510401 16590 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 11:03:29.511152 16590 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:03:29.511662 16583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:29.504576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43393 (local address 127.26.120.254:45335)
0929 11:03:29.504607 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:29.504609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:29.504698 (+ 89us) server_negotiation.cc:407] Connection header received
0929 11:03:29.504761 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:29.504762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:29.504770 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:29.504823 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:29.505431 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:29.506166 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:29.507179 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:29.507486 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:29.507545 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:29.508861 (+ 1316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:29.508862 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:29.508863 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:29.508869 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:29.510480 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:29.510858 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:29.510859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:29.510859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:29.510912 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:29.511108 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:29.511109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:29.511109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:29.511223 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:29.511293 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:29.511498 (+ 205us) server_negotiation.cc:299] Negotiation successful
0929 11:03:29.511580 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260929 11:03:29.514204 16590 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04a8ff76122f40cf87dabc7f44bee1c6: Generated new TSK 0
I20260929 11:03:29.514376 16590 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 11:03:29.516347 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 16511
2026-09-29T11:03:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1315 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 11:03:29 dist-test-slave-2lzq krb5kdc[16610](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 11:03:29 dist-test-slave-2lzq krb5kdc[16610](info): set up 2 sockets
Sep 29 11:03:29 dist-test-slave-2lzq krb5kdc[16610](info): commencing operation
krb5kdc: starting...
W20260929 11:03:30.542373 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 11:03:30 dist-test-slave-2lzq krb5kdc[16610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679810, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T11:03:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T11:03:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.120.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
I20260929 11:03:30.658562 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:34079
--webserver_interface=127.26.120.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:40559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:34079
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:03:30.741142 16623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:30.741389 16623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:30.741430 16623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:30.742590 16623 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:03:30.742664 16623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:30.742689 16623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:30.742717 16623 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:03:30.742748 16623 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:03:30.744201 16623 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:40559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:34079
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:34079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:30.744482 16623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:30.744714 16623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 11:03:30.749536 16623 server_base.cc:1034] running on GCE node
W20260929 11:03:30.749505 16628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:30.749503 16629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:30.749490 16631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:30.749953 16623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:30.750285 16623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:30.751441 16623 hybrid_clock.cc:648] HybridClock initialized: now 1790679810751422 us; error 38 us; skew 500 ppm
Sep 29 11:03:30 dist-test-slave-2lzq krb5kdc[16610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679810, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:03:30.759778 16623 init.cc:377] Logged in from keytab as kudu/127.26.120.254@KRBTEST.COM (short username kudu)
I20260929 11:03:30.760236 16623 webserver.cc:460] Webserver started at http://127.26.120.254:41369/ using document root <none> and password file <none>
I20260929 11:03:30.760453 16623 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:30.760586 16623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:30.760756 16623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:30.761804 16623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "b27e470fb6cd4a5eaf8a98d09b2a5cf4"
format_stamp: "Formatted at 2026-09-29 11:03:30 on dist-test-slave-2lzq"
server_key: "8f1473e5557cb636bbd413408b5b2c88"
server_key_iv: "042214893beb4775ad6816cf82048214"
server_key_version: "encryptionkey@0"
I20260929 11:03:30.762208 16623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "b27e470fb6cd4a5eaf8a98d09b2a5cf4"
format_stamp: "Formatted at 2026-09-29 11:03:30 on dist-test-slave-2lzq"
server_key: "8f1473e5557cb636bbd413408b5b2c88"
server_key_iv: "042214893beb4775ad6816cf82048214"
server_key_version: "encryptionkey@0"
I20260929 11:03:30.763526 16623 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:03:30.764278 16638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:30.764472 16623 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 11:03:30.764570 16623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "b27e470fb6cd4a5eaf8a98d09b2a5cf4"
format_stamp: "Formatted at 2026-09-29 11:03:30 on dist-test-slave-2lzq"
server_key: "8f1473e5557cb636bbd413408b5b2c88"
server_key_iv: "042214893beb4775ad6816cf82048214"
server_key_version: "encryptionkey@0"
I20260929 11:03:30.764657 16623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:30.786793 16623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:30.788678 16623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:30.788848 16623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:30.793246 16623 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:34079
I20260929 11:03:30.793293 16690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:34079 every 8 connection(s)
I20260929 11:03:30.793672 16623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:03:30.794620 16691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:30.797003 16691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b27e470fb6cd4a5eaf8a98d09b2a5cf4: Bootstrap starting.
I20260929 11:03:30.797662 16691 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b27e470fb6cd4a5eaf8a98d09b2a5cf4: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:30.797964 16691 log.cc:824] T 00000000000000000000000000000000 P b27e470fb6cd4a5eaf8a98d09b2a5cf4: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:30.798722 16691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b27e470fb6cd4a5eaf8a98d09b2a5cf4: No bootstrap required, opened a new log
I20260929 11:03:30.800323 16691 raft_consensus.cc:348] T 00000000000000000000000000000000 P b27e470fb6cd4a5eaf8a98d09b2a5cf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b27e470fb6cd4a5eaf8a98d09b2a5cf4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34079 } }
I20260929 11:03:30.800469 16691 raft_consensus.cc:374] T 00000000000000000000000000000000 P b27e470fb6cd4a5eaf8a98d09b2a5cf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:30.800493 16691 raft_consensus.cc:729] T 00000000000000000000000000000000 P b27e470fb6cd4a5eaf8a98d09b2a5cf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b27e470fb6cd4a5eaf8a98d09b2a5cf4, State: Initialized, Role: FOLLOWER
I20260929 11:03:30.800683 16691 consensus_queue.cc:260] T 00000000000000000000000000000000 P b27e470fb6cd4a5eaf8a98d09b2a5cf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b27e470fb6cd4a5eaf8a98d09b2a5cf4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34079 } }
I20260929 11:03:30.800750 16691 raft_consensus.cc:388] T 00000000000000000000000000000000 P b27e470fb6cd4a5eaf8a98d09b2a5cf4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 11:03:30.800771 16691 raft_consensus.cc:482] T 00000000000000000000000000000000 P b27e470fb6cd4a5eaf8a98d09b2a5cf4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 11:03:30.800830 16691 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b27e470fb6cd4a5eaf8a98d09b2a5cf4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:30.801705 16691 raft_consensus.cc:504] T 00000000000000000000000000000000 P b27e470fb6cd4a5eaf8a98d09b2a5cf4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b27e470fb6cd4a5eaf8a98d09b2a5cf4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34079 } }
I20260929 11:03:30.801816 16691 leader_election.cc:302] T 00000000000000000000000000000000 P b27e470fb6cd4a5eaf8a98d09b2a5cf4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b27e470fb6cd4a5eaf8a98d09b2a5cf4; no voters:
I20260929 11:03:30.802002 16691 leader_election.cc:288] T 00000000000000000000000000000000 P b27e470fb6cd4a5eaf8a98d09b2a5cf4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 11:03:30.802084 16694 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b27e470fb6cd4a5eaf8a98d09b2a5cf4 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:03:30.802253 16694 raft_consensus.cc:686] T 00000000000000000000000000000000 P b27e470fb6cd4a5eaf8a98d09b2a5cf4 [term 1 LEADER]: Becoming Leader. State: Replica: b27e470fb6cd4a5eaf8a98d09b2a5cf4, State: Running, Role: LEADER
I20260929 11:03:30.802299 16691 sys_catalog.cc:564] T 00000000000000000000000000000000 P b27e470fb6cd4a5eaf8a98d09b2a5cf4 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:03:30.802376 16694 consensus_queue.cc:237] T 00000000000000000000000000000000 P b27e470fb6cd4a5eaf8a98d09b2a5cf4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b27e470fb6cd4a5eaf8a98d09b2a5cf4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34079 } }
I20260929 11:03:30.802810 16695 sys_catalog.cc:455] T 00000000000000000000000000000000 P b27e470fb6cd4a5eaf8a98d09b2a5cf4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b27e470fb6cd4a5eaf8a98d09b2a5cf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b27e470fb6cd4a5eaf8a98d09b2a5cf4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34079 } } }
I20260929 11:03:30.803131 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 16623
I20260929 11:03:30.803252 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 11:03:30.803462 27107 external_mini_cluster.cc:1428] Setting key a53e59cf7f569c1c91fe396aa17106a2
I20260929 11:03:30.802821 16696 sys_catalog.cc:455] T 00000000000000000000000000000000 P b27e470fb6cd4a5eaf8a98d09b2a5cf4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b27e470fb6cd4a5eaf8a98d09b2a5cf4. Latest consensus state: current_term: 1 leader_uuid: "b27e470fb6cd4a5eaf8a98d09b2a5cf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b27e470fb6cd4a5eaf8a98d09b2a5cf4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 34079 } } }
I20260929 11:03:30.803663 16696 sys_catalog.cc:458] T 00000000000000000000000000000000 P b27e470fb6cd4a5eaf8a98d09b2a5cf4 [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:30.803798 16695 sys_catalog.cc:458] T 00000000000000000000000000000000 P b27e470fb6cd4a5eaf8a98d09b2a5cf4 [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:30.803937 16710 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:03:30.804437 16710 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:03:30.805998 16710 catalog_manager.cc:1347] Generated new cluster ID: 2c77bf4c89784432a5b4bcc62e69d0f9
I20260929 11:03:30.806072 16710 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 11:03:30.809145 16713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:30.804190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41381 (local address 127.26.120.254:34079)
0929 11:03:30.804409 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:30.804418 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:30.804450 (+ 32us) server_negotiation.cc:407] Connection header received
0929 11:03:30.804605 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:30.804616 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:30.804661 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:30.804799 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:30.805024 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:30.806398 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:30.807184 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:30.807618 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:30.807744 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:30.808471 (+ 727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:30.808475 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:30.808475 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:30.808660 (+ 185us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:30.808692 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:30.808703 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:30.808779 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:30.808851 (+ 72us) server_negotiation.cc:299] Negotiation successful
0929 11:03:30.808915 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":92,"threads_started":1}
I20260929 11:03:30.810009 27107 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 11:03:30.814221 16713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:30.811387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44195 (local address 127.26.120.254:34079)
0929 11:03:30.811428 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:30.811430 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:30.811553 (+ 123us) server_negotiation.cc:407] Connection header received
0929 11:03:30.811624 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:30.811625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:30.811636 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:30.811691 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:30.811810 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:30.812292 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:30.812982 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:30.813192 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:30.813278 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:30.813968 (+ 690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:30.813969 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:30.813969 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:30.813974 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:30.813988 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:30.813992 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:30.814039 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:30.814120 (+ 81us) server_negotiation.cc:299] Negotiation successful
0929 11:03:30.814146 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 11:03:30.826213 16710 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 11:03:30.826799 16710 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:03:30.832099 16710 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b27e470fb6cd4a5eaf8a98d09b2a5cf4: Generated new TSK 0
I20260929 11:03:30.832288 16710 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 11:03:30.834448 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 16623
2026-09-29T11:03:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1317 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 11:03:30 dist-test-slave-2lzq krb5kdc[16722](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 11:03:30 dist-test-slave-2lzq krb5kdc[16722](info): set up 2 sockets
Sep 29 11:03:30 dist-test-slave-2lzq krb5kdc[16722](info): commencing operation
krb5kdc: starting...
W20260929 11:03:31.902521 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 11:03:31 dist-test-slave-2lzq krb5kdc[16722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T11:03:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T11:03:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.120.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
I20260929 11:03:32.020529 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:37845
--webserver_interface=127.26.120.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:41515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:37845
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:03:32.102397 16735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:32.102639 16735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:32.102677 16735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:32.103859 16735 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:03:32.103927 16735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:32.103951 16735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:32.103981 16735 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:03:32.104012 16735 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:03:32.105621 16735 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:41515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:37845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:37845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:32.105914 16735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:32.106225 16735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:32.111042 16743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:32.111042 16741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:32.111091 16740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:32.111485 16735 server_base.cc:1034] running on GCE node
I20260929 11:03:32.111714 16735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:32.111963 16735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:32.113107 16735 hybrid_clock.cc:648] HybridClock initialized: now 1790679812113093 us; error 29 us; skew 500 ppm
Sep 29 11:03:32 dist-test-slave-2lzq krb5kdc[16722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679812, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:03:32.115725 16735 init.cc:377] Logged in from keytab as kudu/127.26.120.254@KRBTEST.COM (short username kudu)
I20260929 11:03:32.116137 16735 webserver.cc:460] Webserver started at http://127.26.120.254:45125/ using document root <none> and password file <none>
I20260929 11:03:32.116354 16735 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:32.116400 16735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:32.116492 16735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:32.117539 16735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "c9858bc5b08448c6b072fc997ae6e7d5"
format_stamp: "Formatted at 2026-09-29 11:03:32 on dist-test-slave-2lzq"
server_key: "d1c511f86a9e60678c897cd397604e1e"
server_key_iv: "dd37a023300b919eb2625f719989fc7c"
server_key_version: "encryptionkey@0"
I20260929 11:03:32.117847 16735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "c9858bc5b08448c6b072fc997ae6e7d5"
format_stamp: "Formatted at 2026-09-29 11:03:32 on dist-test-slave-2lzq"
server_key: "d1c511f86a9e60678c897cd397604e1e"
server_key_iv: "dd37a023300b919eb2625f719989fc7c"
server_key_version: "encryptionkey@0"
I20260929 11:03:32.119385 16735 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 11:03:32.120257 16750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:32.120496 16735 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 11:03:32.120620 16735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "c9858bc5b08448c6b072fc997ae6e7d5"
format_stamp: "Formatted at 2026-09-29 11:03:32 on dist-test-slave-2lzq"
server_key: "d1c511f86a9e60678c897cd397604e1e"
server_key_iv: "dd37a023300b919eb2625f719989fc7c"
server_key_version: "encryptionkey@0"
I20260929 11:03:32.120708 16735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:32.124320 16735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:32.126775 16735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:32.126943 16735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:32.131009 16735 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:37845
I20260929 11:03:32.131147 16802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:37845 every 8 connection(s)
I20260929 11:03:32.131443 16735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:03:32.132470 16803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:32.134853 16803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9858bc5b08448c6b072fc997ae6e7d5: Bootstrap starting.
I20260929 11:03:32.135154 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 16735
I20260929 11:03:32.135282 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 11:03:32.135488 27107 external_mini_cluster.cc:1428] Setting key fbef3bd240b44a4da6a356f9bd4a6434
I20260929 11:03:32.135643 16803 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9858bc5b08448c6b072fc997ae6e7d5: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:32.136023 16803 log.cc:824] T 00000000000000000000000000000000 P c9858bc5b08448c6b072fc997ae6e7d5: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:32.136752 16803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9858bc5b08448c6b072fc997ae6e7d5: No bootstrap required, opened a new log
I20260929 11:03:32.138188 16806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:32.136048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51507 (local address 127.26.120.254:37845)
0929 11:03:32.136249 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:32.136259 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:32.136295 (+ 36us) server_negotiation.cc:407] Connection header received
0929 11:03:32.136880 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:32.136888 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:32.136916 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:32.137037 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:32.137694 (+ 657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:32.137697 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:32.137697 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:32.137700 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:32.137738 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:32.137748 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:32.137814 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:32.137880 (+ 66us) server_negotiation.cc:299] Negotiation successful
0929 11:03:32.137936 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":98,"threads_started":1}
I20260929 11:03:32.139201 27107 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 11:03:32.139465 16803 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9858bc5b08448c6b072fc997ae6e7d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9858bc5b08448c6b072fc997ae6e7d5" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 37845 } }
I20260929 11:03:32.139601 16803 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9858bc5b08448c6b072fc997ae6e7d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 11:03:32.139623 16803 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9858bc5b08448c6b072fc997ae6e7d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9858bc5b08448c6b072fc997ae6e7d5, State: Initialized, Role: FOLLOWER
I20260929 11:03:32.139761 16803 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9858bc5b08448c6b072fc997ae6e7d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9858bc5b08448c6b072fc997ae6e7d5" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 37845 } }
I20260929 11:03:32.139840 16803 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9858bc5b08448c6b072fc997ae6e7d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 11:03:32.139884 16803 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9858bc5b08448c6b072fc997ae6e7d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 11:03:32.139935 16803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9858bc5b08448c6b072fc997ae6e7d5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:32.140599 16803 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9858bc5b08448c6b072fc997ae6e7d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9858bc5b08448c6b072fc997ae6e7d5" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 37845 } }
I20260929 11:03:32.140719 16803 leader_election.cc:302] T 00000000000000000000000000000000 P c9858bc5b08448c6b072fc997ae6e7d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9858bc5b08448c6b072fc997ae6e7d5; no voters:
I20260929 11:03:32.140964 16803 leader_election.cc:288] T 00000000000000000000000000000000 P c9858bc5b08448c6b072fc997ae6e7d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 11:03:32.141032 16813 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9858bc5b08448c6b072fc997ae6e7d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:03:32.141188 16813 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9858bc5b08448c6b072fc997ae6e7d5 [term 1 LEADER]: Becoming Leader. State: Replica: c9858bc5b08448c6b072fc997ae6e7d5, State: Running, Role: LEADER
I20260929 11:03:32.141265 16803 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9858bc5b08448c6b072fc997ae6e7d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:03:32.141342 16813 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9858bc5b08448c6b072fc997ae6e7d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9858bc5b08448c6b072fc997ae6e7d5" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 37845 } }
I20260929 11:03:32.141769 16813 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9858bc5b08448c6b072fc997ae6e7d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9858bc5b08448c6b072fc997ae6e7d5. Latest consensus state: current_term: 1 leader_uuid: "c9858bc5b08448c6b072fc997ae6e7d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9858bc5b08448c6b072fc997ae6e7d5" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 37845 } } }
I20260929 11:03:32.141912 16813 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9858bc5b08448c6b072fc997ae6e7d5 [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:32.141872 16815 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9858bc5b08448c6b072fc997ae6e7d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9858bc5b08448c6b072fc997ae6e7d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9858bc5b08448c6b072fc997ae6e7d5" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 37845 } } }
I20260929 11:03:32.142104 16806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:32.140734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52365 (local address 127.26.120.254:37845)
0929 11:03:32.140771 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:32.140774 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:32.140905 (+ 131us) server_negotiation.cc:407] Connection header received
0929 11:03:32.140939 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:32.140940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:32.140946 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:32.141008 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:32.141664 (+ 656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:32.141665 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:32.141666 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 11:03:32.141672 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:32.141687 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 11:03:32.141692 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:32.141754 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:32.142003 (+ 249us) server_negotiation.cc:299] Negotiation successful
0929 11:03:32.142022 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 11:03:32.142391 16815 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9858bc5b08448c6b072fc997ae6e7d5 [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:32.142652 16822 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:03:32.143186 16822 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:03:32.144613 16822 catalog_manager.cc:1347] Generated new cluster ID: 3bee6930d7784de09a7aae4e30d8c73e
I20260929 11:03:32.144665 16822 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 11:03:32.154615 16822 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 11:03:32.154863 16822 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:03:32.163829 16822 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9858bc5b08448c6b072fc997ae6e7d5: Generated new TSK 0
I20260929 11:03:32.164042 16822 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 11:03:32.169776 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 16735
2026-09-29T11:03:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1335 ms)
[----------] 18 tests from AuthTokenIssuingTest (23627 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 11:03:32 dist-test-slave-2lzq krb5kdc[16834](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 11:03:32 dist-test-slave-2lzq krb5kdc[16834](info): set up 2 sockets
Sep 29 11:03:32 dist-test-slave-2lzq krb5kdc[16834](info): commencing operation
krb5kdc: starting...
W20260929 11:03:33.202481 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 11:03:33 dist-test-slave-2lzq krb5kdc[16834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T11:03:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T11:03:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.120.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
I20260929 11:03:33.319233 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:33691
--webserver_interface=127.26.120.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:35281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:33691,127.26.120.253:44909,127.26.120.252:43167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:03:33.399168 16847 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 11:03:33.399439 16847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:33.399469 16847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:33.399518 16847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:33.400695 16847 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:03:33.400770 16847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:33.400795 16847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:33.400826 16847 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:03:33.400877 16847 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:03:33.402493 16847 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:35281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:33691,127.26.120.253:44909,127.26.120.252:43167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:33691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:33.402768 16847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:33.403007 16847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:33.408025 16852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:33.408056 16855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:33.408174 16853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:33.408396 16847 server_base.cc:1034] running on GCE node
I20260929 11:03:33.408655 16847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:33.408952 16847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:33.410127 16847 hybrid_clock.cc:648] HybridClock initialized: now 1790679813410077 us; error 55 us; skew 500 ppm
Sep 29 11:03:33 dist-test-slave-2lzq krb5kdc[16834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679813, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:03:33.412560 16847 init.cc:377] Logged in from keytab as kudu/127.26.120.254@KRBTEST.COM (short username kudu)
I20260929 11:03:33.413036 16847 webserver.cc:460] Webserver started at http://127.26.120.254:42129/ using document root <none> and password file <none>
I20260929 11:03:33.413288 16847 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:33.413357 16847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:33.413507 16847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:33.414633 16847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "5e254883532840fbb608428bfeef47ef"
format_stamp: "Formatted at 2026-09-29 11:03:33 on dist-test-slave-2lzq"
server_key: "9c71be14b47a63fc3a592a2f5ae340f3"
server_key_iv: "e1105ba7360c05e00cc0d8f380bc11b3"
server_key_version: "encryptionkey@0"
I20260929 11:03:33.414994 16847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "5e254883532840fbb608428bfeef47ef"
format_stamp: "Formatted at 2026-09-29 11:03:33 on dist-test-slave-2lzq"
server_key: "9c71be14b47a63fc3a592a2f5ae340f3"
server_key_iv: "e1105ba7360c05e00cc0d8f380bc11b3"
server_key_version: "encryptionkey@0"
I20260929 11:03:33.416404 16847 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 11:03:33.417171 16862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:33.417383 16847 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:03:33.417493 16847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "5e254883532840fbb608428bfeef47ef"
format_stamp: "Formatted at 2026-09-29 11:03:33 on dist-test-slave-2lzq"
server_key: "9c71be14b47a63fc3a592a2f5ae340f3"
server_key_iv: "e1105ba7360c05e00cc0d8f380bc11b3"
server_key_version: "encryptionkey@0"
I20260929 11:03:33.417567 16847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:33.430452 16847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:33.432547 16847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:33.432718 16847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:33.436908 16847 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:33691
I20260929 11:03:33.436942 16914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:33691 every 8 connection(s)
I20260929 11:03:33.437309 16847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:03:33.438246 16915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:33.439729 16915 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.120.254" port: 33691 } has no permanent_uuid. Determining permanent_uuid...
I20260929 11:03:33.443964 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 16847
I20260929 11:03:33.444079 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 11:03:33.444320 27107 external_mini_cluster.cc:1428] Setting key b65b943e9e5049d61073000570c96ad9
Sep 29 11:03:33 dist-test-slave-2lzq krb5kdc[16834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679813, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:03:33.449668 16917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.440375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:33691 (local address 127.0.0.1:57880)
0929 11:03:33.440536 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:33.440578 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:33.441032 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:33.441303 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:33.441306 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:33.441564 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:03:33.441730 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:33.441734 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.442809 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.442810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:33.443495 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:33.443497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.443957 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.443958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:33.444050 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:33.444670 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:03:33.444678 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:03:33.446298 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:03:33.448144 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:33.448150 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:33.448152 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:33.448347 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:33.448608 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:33.448609 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:33.448609 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:33.448695 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:33.449062 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:03:33.449067 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:03:33.449132 (+ 65us) client_negotiation.cc:769] Sending connection context
0929 11:03:33.449238 (+ 106us) client_negotiation.cc:240] Negotiation successful
0929 11:03:33.449490 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":108,"thread_start_us":88,"threads_started":1}
I20260929 11:03:33.449672 16918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.440424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57880 (local address 127.26.120.254:33691)
0929 11:03:33.440580 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:33.440586 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:33.440674 (+ 88us) server_negotiation.cc:407] Connection header received
0929 11:03:33.441154 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:33.441156 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:33.441173 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:33.441258 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:33.441817 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.442719 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.443567 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.443867 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.443946 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:33.446381 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:33.446389 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:33.446390 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:33.446398 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:33.448042 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:33.448446 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:33.448449 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:33.448449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:33.448532 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:33.448762 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:33.448763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:33.448763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:33.448983 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:33.449070 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:33.449271 (+ 201us) server_negotiation.cc:299] Negotiation successful
0929 11:03:33.449426 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":89,"threads_started":1}
I20260929 11:03:33.450502 16915 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.120.253" port: 44909 } has no permanent_uuid. Determining permanent_uuid...
I20260929 11:03:33.450853 16917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.450746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.253:44909 (local address 127.0.0.1:40668)
0929 11:03:33.450791 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:33.450834 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.253:44909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260929 11:03:33.450994 16863 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.120.253:44909: connect: Connection refused (error 111)
I20260929 11:03:33.451409 16917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.451301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.253:44909 (local address 127.0.0.1:40682)
0929 11:03:33.451348 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:33.451380 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.253:44909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260929 11:03:33.451557 16915 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.120.253:44909: Network error: Client connection negotiation failed: client connection to 127.26.120.253:44909: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.26.120.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.253.keytab.
Entry for principal kudu/127.26.120.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.253.keytab.
I20260929 11:03:33.469096 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.253:44909
--webserver_interface=127.26.120.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.253.keytab
--principal=kudu/127.26.120.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:35281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:33691,127.26.120.253:44909,127.26.120.252:43167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 11:03:33.500820 16915 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.120.253" port: 44909 } attempt: 1
I20260929 11:03:33.501386 16917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.501220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.253:44909 (local address 127.0.0.1:40684)
0929 11:03:33.501293 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:33.501345 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.253:44909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260929 11:03:33.501798 16917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.501701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.253:44909 (local address 127.0.0.1:40696)
0929 11:03:33.501748 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:33.501778 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.253:44909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260929 11:03:33.501936 16915 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.120.253:44909: Network error: Client connection negotiation failed: client connection to 127.26.120.253:44909: connect: Connection refused (error 111)
W20260929 11:03:33.550912 16922 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 11:03:33.551120 16922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:33.551138 16922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:33.551158 16922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:33.552361 16922 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:03:33.552403 16922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:33.552418 16922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:33.552455 16922 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:03:33.552482 16922 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:03:33.554018 16922 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:35281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:33691,127.26.120.253:44909,127.26.120.252:43167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.253.keytab
--principal=kudu/127.26.120.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.253:44909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.120.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:33.554315 16922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:33.554559 16922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:33.559589 16930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:33.559576 16928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:33.559578 16927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:33.559676 16922 server_base.cc:1034] running on GCE node
I20260929 11:03:33.560034 16922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:33.560344 16922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:33.561544 16922 hybrid_clock.cc:648] HybridClock initialized: now 1790679813561531 us; error 32 us; skew 500 ppm
Sep 29 11:03:33 dist-test-slave-2lzq krb5kdc[16834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679813, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:03:33.564462 16922 init.cc:377] Logged in from keytab as kudu/127.26.120.253@KRBTEST.COM (short username kudu)
I20260929 11:03:33.564949 16922 webserver.cc:460] Webserver started at http://127.26.120.253:35493/ using document root <none> and password file <none>
I20260929 11:03:33.565209 16922 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:33.565286 16922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:33.565442 16922 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:33.566669 16922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-1/data/instance:
uuid: "e0c862ca18a84ef8a8895b2334a74692"
format_stamp: "Formatted at 2026-09-29 11:03:33 on dist-test-slave-2lzq"
server_key: "4dd26620d3fda83276b211f90604f515"
server_key_iv: "bb0c231ed6e7a33ee2b90293a41b6525"
server_key_version: "encryptionkey@0"
I20260929 11:03:33.567057 16922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-1/wal/instance:
uuid: "e0c862ca18a84ef8a8895b2334a74692"
format_stamp: "Formatted at 2026-09-29 11:03:33 on dist-test-slave-2lzq"
server_key: "4dd26620d3fda83276b211f90604f515"
server_key_iv: "bb0c231ed6e7a33ee2b90293a41b6525"
server_key_version: "encryptionkey@0"
I20260929 11:03:33.568677 16922 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 11:03:33.569435 16937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:33.569609 16922 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 11:03:33.569720 16922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-1/wal
uuid: "e0c862ca18a84ef8a8895b2334a74692"
format_stamp: "Formatted at 2026-09-29 11:03:33 on dist-test-slave-2lzq"
server_key: "4dd26620d3fda83276b211f90604f515"
server_key_iv: "bb0c231ed6e7a33ee2b90293a41b6525"
server_key_version: "encryptionkey@0"
I20260929 11:03:33.569798 16922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:33.570117 16915 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.120.253" port: 44909 } attempt: 2
I20260929 11:03:33.570544 16917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.570389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.253:44909 (local address 127.0.0.1:40698)
0929 11:03:33.570454 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:33.570499 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.253:44909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260929 11:03:33.570910 16917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.570811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.253:44909 (local address 127.0.0.1:40712)
0929 11:03:33.570859 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:33.570888 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.253:44909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260929 11:03:33.571059 16915 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.120.253:44909: Network error: Client connection negotiation failed: client connection to 127.26.120.253:44909: connect: Connection refused (error 111)
I20260929 11:03:33.596518 16922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:33.599015 16922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:33.599154 16922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:33.603118 16922 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.253:44909
I20260929 11:03:33.603158 16989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.253:44909 every 8 connection(s)
I20260929 11:03:33.603543 16922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-1/data/info.pb
I20260929 11:03:33.604084 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 16922
I20260929 11:03:33.604193 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-1/wal/instance
I20260929 11:03:33.604398 27107 external_mini_cluster.cc:1428] Setting key 67f84c0af9d782185c983bd32c2edf3f
I20260929 11:03:33.604642 16990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:33.606736 16990 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.120.254" port: 33691 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.26.120.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.252@KRBTEST.COM" created.
Sep 29 11:03:33 dist-test-slave-2lzq krb5kdc[16834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679813, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.253@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:03:33.619419 16918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.607458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57894 (local address 127.26.120.254:33691)
0929 11:03:33.607533 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:33.607535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:33.607804 (+ 269us) server_negotiation.cc:407] Connection header received
0929 11:03:33.608561 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:33.608562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:33.608575 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:33.608627 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:33.609494 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.609984 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.611507 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.611705 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.611759 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:33.616267 (+ 4508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:33.616269 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:33.616269 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:33.616278 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:33.617914 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:33.618348 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:33.618350 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:33.618351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:33.618414 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:33.618659 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:33.618660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:33.618661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:33.618817 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:33.618891 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:33.619222 (+ 331us) server_negotiation.cc:299] Negotiation successful
0929 11:03:33.619312 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 11:03:33.619426 16993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.607441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:33691 (local address 127.0.0.1:57894)
0929 11:03:33.607654 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:33.607705 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:33.608450 (+ 745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:33.608724 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:33.608728 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:33.609107 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:03:33.609405 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:33.609412 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.610100 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.610101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:33.611421 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:33.611425 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.611799 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.611801 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:33.611958 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:33.612458 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:03:33.612469 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:03:33.616166 (+ 3697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:03:33.617991 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:33.617997 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:33.617999 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:33.618275 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:33.618482 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:33.618483 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:33.618483 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:33.618580 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:33.618886 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:03:33.618909 (+ 23us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:03:33.619060 (+ 151us) client_negotiation.cc:769] Sending connection context
0929 11:03:33.619141 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 11:03:33.619260 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":100,"threads_started":1}
I20260929 11:03:33.620065 16990 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.120.253" port: 44909 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 11:03:33 dist-test-slave-2lzq krb5kdc[16834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679813, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.253@KRBTEST.COM for kudu/127.26.120.253@KRBTEST.COM
I20260929 11:03:33.628796 16993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.620359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.253:44909 (local address 127.0.0.1:40728)
0929 11:03:33.620424 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:33.620433 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:33.620474 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:33.620724 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:33.620724 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:33.620874 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:03:33.620918 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:33.620920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.621871 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.621872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:33.622441 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:33.622443 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.622936 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.622937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:33.623001 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:33.623421 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:03:33.623425 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:03:33.626023 (+ 2598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:03:33.627706 (+ 1683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:33.627708 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:33.627708 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:33.627877 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:33.628097 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:33.628098 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:33.628098 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:33.628155 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:33.628454 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:03:33.628455 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:03:33.628493 (+ 38us) client_negotiation.cc:769] Sending connection context
0929 11:03:33.628564 (+ 71us) client_negotiation.cc:240] Negotiation successful
0929 11:03:33.628661 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260929 11:03:33.628834 16995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.620442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40728 (local address 127.26.120.253:44909)
0929 11:03:33.620596 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:33.620600 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:33.620615 (+ 15us) server_negotiation.cc:407] Connection header received
0929 11:03:33.620631 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:33.620633 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:33.620643 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:33.620724 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:33.620994 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.621775 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.622531 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.622838 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.622937 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:33.626209 (+ 3272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:33.626217 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:33.626218 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:33.626224 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:33.627618 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:33.627970 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:33.627974 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:33.627974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:33.628029 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:33.628213 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:33.628214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:33.628214 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:33.628377 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:33.628476 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:33.628593 (+ 117us) server_negotiation.cc:299] Negotiation successful
0929 11:03:33.628712 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":131,"thread_start_us":81,"threads_started":1}
I20260929 11:03:33.629537 16990 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.120.252" port: 43167 } has no permanent_uuid. Determining permanent_uuid...
I20260929 11:03:33.629868 16993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.629760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:43167 (local address 127.0.0.1:45980)
0929 11:03:33.629794 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:33.629848 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.252:43167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260929 11:03:33.630010 16941 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.120.252:43167: connect: Connection refused (error 111)
I20260929 11:03:33.630471 16993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.630384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:43167 (local address 127.0.0.1:45984)
0929 11:03:33.630431 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:33.630455 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.252:43167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260929 11:03:33.630640 16990 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.120.252:43167: Network error: Client connection negotiation failed: client connection to 127.26.120.252:43167: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.252.keytab.
Entry for principal kudu/127.26.120.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.252.keytab.
I20260929 11:03:33.632267 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.252:43167
--webserver_interface=127.26.120.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.252.keytab
--principal=kudu/127.26.120.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:35281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:33691,127.26.120.253:44909,127.26.120.252:43167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 11:03:33.662257 16915 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.120.253" port: 44909 } attempt: 3
Sep 29 11:03:33 dist-test-slave-2lzq krb5kdc[16834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679813, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for kudu/127.26.120.253@KRBTEST.COM
I20260929 11:03:33.669518 16917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.662640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.253:44909 (local address 127.0.0.1:40732)
0929 11:03:33.662694 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:33.662713 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:33.662788 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:33.662943 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:33.662944 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:33.663107 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:03:33.663174 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:33.663176 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.663847 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.663849 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:33.664395 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:33.664397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.664792 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.664793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:33.664879 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:33.665379 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:03:33.665384 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:03:33.666658 (+ 1274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:03:33.668535 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:33.668536 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:33.668537 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:33.668697 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:33.668931 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:33.668931 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:33.668932 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:33.668983 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:33.669290 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:03:33.669290 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:03:33.669329 (+ 39us) client_negotiation.cc:769] Sending connection context
0929 11:03:33.669377 (+ 48us) client_negotiation.cc:240] Negotiation successful
0929 11:03:33.669449 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260929 11:03:33.669644 16995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.662720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40732 (local address 127.26.120.253:44909)
0929 11:03:33.662800 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:33.662802 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:33.662815 (+ 13us) server_negotiation.cc:407] Connection header received
0929 11:03:33.662848 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:33.662849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:33.662860 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:33.662938 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:33.663264 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.663758 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.664489 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.664715 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.664796 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:33.666775 (+ 1979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:33.666777 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:33.666777 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:33.666785 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:33.668441 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:33.668776 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:33.668777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:33.668778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:33.668855 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:33.669048 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:33.669049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:33.669050 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:33.669214 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:33.669292 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:33.669418 (+ 126us) server_negotiation.cc:299] Negotiation successful
0929 11:03:33.669516 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 11:03:33.670042 16915 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.120.252" port: 43167 } has no permanent_uuid. Determining permanent_uuid...
I20260929 11:03:33.670481 16917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.670373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:43167 (local address 127.0.0.1:45996)
0929 11:03:33.670427 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:33.670455 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.252:43167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260929 11:03:33.670851 16917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.670760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:43167 (local address 127.0.0.1:46006)
0929 11:03:33.670808 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:33.670831 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.252:43167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260929 11:03:33.670976 16915 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.120.252:43167: Network error: Client connection negotiation failed: client connection to 127.26.120.252:43167: connect: Connection refused (error 111)
I20260929 11:03:33.679893 16990 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.120.252" port: 43167 } attempt: 1
I20260929 11:03:33.680269 16993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.680153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:43167 (local address 127.0.0.1:46022)
0929 11:03:33.680208 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:33.680243 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.252:43167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260929 11:03:33.680583 16993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.680485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:43167 (local address 127.0.0.1:46036)
0929 11:03:33.680530 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:33.680554 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.252:43167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260929 11:03:33.680672 16990 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.120.252:43167: Network error: Client connection negotiation failed: client connection to 127.26.120.252:43167: connect: Connection refused (error 111)
I20260929 11:03:33.702147 16915 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.120.252" port: 43167 } attempt: 1
I20260929 11:03:33.702590 16917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.702456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:43167 (local address 127.0.0.1:46050)
0929 11:03:33.702524 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:33.702562 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.252:43167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260929 11:03:33.702924 16917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.702831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:43167 (local address 127.0.0.1:46058)
0929 11:03:33.702877 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:33.702900 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.252:43167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260929 11:03:33.703015 16915 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.120.252:43167: Network error: Client connection negotiation failed: client connection to 127.26.120.252:43167: connect: Connection refused (error 111)
W20260929 11:03:33.715127 16997 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 11:03:33.715335 16997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:33.715353 16997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:33.715404 16997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:33.716605 16997 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:03:33.716643 16997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:33.716657 16997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:33.716692 16997 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:03:33.716718 16997 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:03:33.718255 16997 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:35281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:33691,127.26.120.253:44909,127.26.120.252:43167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.252.keytab
--principal=kudu/127.26.120.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.252:43167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.120.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:33.718497 16997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:33.718737 16997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:33.723733 17003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:33.723752 16997 server_base.cc:1034] running on GCE node
W20260929 11:03:33.723912 17005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:33.723780 17002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:33.724303 16997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:33.724612 16997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:33.725800 16997 hybrid_clock.cc:648] HybridClock initialized: now 1790679813725784 us; error 31 us; skew 500 ppm
Sep 29 11:03:33 dist-test-slave-2lzq krb5kdc[16834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679813, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:03:33.728350 16997 init.cc:377] Logged in from keytab as kudu/127.26.120.252@KRBTEST.COM (short username kudu)
I20260929 11:03:33.728801 16997 webserver.cc:460] Webserver started at http://127.26.120.252:41559/ using document root <none> and password file <none>
I20260929 11:03:33.729027 16997 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:33.729071 16997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:33.729246 16997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:33.730371 16997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-2/data/instance:
uuid: "f3af9fff7b6949a48f898b8e799c546c"
format_stamp: "Formatted at 2026-09-29 11:03:33 on dist-test-slave-2lzq"
server_key: "27a5636fac49d94b1bb62f334d907573"
server_key_iv: "0876c064457402eebeb9a80df10077ed"
server_key_version: "encryptionkey@0"
I20260929 11:03:33.730726 16997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-2/wal/instance:
uuid: "f3af9fff7b6949a48f898b8e799c546c"
format_stamp: "Formatted at 2026-09-29 11:03:33 on dist-test-slave-2lzq"
server_key: "27a5636fac49d94b1bb62f334d907573"
server_key_iv: "0876c064457402eebeb9a80df10077ed"
server_key_version: "encryptionkey@0"
I20260929 11:03:33.732028 16997 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:03:33.732745 17012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:33.732926 16997 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 11:03:33.733026 16997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-2/wal
uuid: "f3af9fff7b6949a48f898b8e799c546c"
format_stamp: "Formatted at 2026-09-29 11:03:33 on dist-test-slave-2lzq"
server_key: "27a5636fac49d94b1bb62f334d907573"
server_key_iv: "0876c064457402eebeb9a80df10077ed"
server_key_version: "encryptionkey@0"
I20260929 11:03:33.733107 16997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:33.748824 16990 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.120.252" port: 43167 } attempt: 2
I20260929 11:03:33.749281 16993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.749131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:43167 (local address 127.0.0.1:46062)
0929 11:03:33.749197 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:33.749242 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.252:43167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260929 11:03:33.749679 16993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.749579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:43167 (local address 127.0.0.1:46070)
0929 11:03:33.749629 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:33.749657 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.252:43167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260929 11:03:33.749799 16990 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.120.252:43167: Network error: Client connection negotiation failed: client connection to 127.26.120.252:43167: connect: Connection refused (error 111)
I20260929 11:03:33.751695 16997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:33.753484 16997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:33.753600 16997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:33.757845 16997 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.252:43167
I20260929 11:03:33.757879 17064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.252:43167 every 8 connection(s)
I20260929 11:03:33.758296 16997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-2/data/info.pb
I20260929 11:03:33.759109 17065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:33.760618 17065 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.120.254" port: 33691 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 11:03:33 dist-test-slave-2lzq krb5kdc[16834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679813, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.252@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:03:33.767244 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 16997
I20260929 11:03:33.767359 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790679549986039-27107-0/minicluster-data/master-2/wal/instance
I20260929 11:03:33.767580 27107 external_mini_cluster.cc:1428] Setting key 0d8f49458663f361319c051967ba5f59
I20260929 11:03:33.769974 16918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.761304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57906 (local address 127.26.120.254:33691)
0929 11:03:33.761395 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:33.761398 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:33.761570 (+ 172us) server_negotiation.cc:407] Connection header received
0929 11:03:33.762115 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:33.762116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:33.762129 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:33.762184 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:33.762814 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.763283 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.764347 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.764553 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.764629 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:33.766884 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:33.766885 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:33.766885 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:33.766893 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:33.768543 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:33.769037 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:33.769038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:33.769039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:33.769108 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:33.769355 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:33.769356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:33.769356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:33.769494 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:33.769564 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:33.769799 (+ 235us) server_negotiation.cc:299] Negotiation successful
0929 11:03:33.769878 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 11:03:33.770357 17067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.761284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:33691 (local address 127.0.0.1:57906)
0929 11:03:33.761448 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:33.761492 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:33.761980 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:33.762269 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:33.762277 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:33.762524 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:03:33.762710 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:33.762714 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.763385 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.763386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:33.764246 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:33.764249 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.764634 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.764635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:33.764789 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:33.765295 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:03:33.765302 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:03:33.766791 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:03:33.768623 (+ 1832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:33.768629 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:33.768631 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:33.768937 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:33.769184 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:33.769185 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:33.769185 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:33.769285 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:33.769561 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:03:33.769566 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:03:33.769673 (+ 107us) client_negotiation.cc:769] Sending connection context
0929 11:03:33.769753 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 11:03:33.770138 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":183,"thread_start_us":71,"threads_started":1}
I20260929 11:03:33.770939 17065 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.120.253" port: 44909 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 11:03:33 dist-test-slave-2lzq krb5kdc[16834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.252@KRBTEST.COM
Sep 29 11:03:33 dist-test-slave-2lzq krb5kdc[16834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679813, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.252@KRBTEST.COM for kudu/127.26.120.253@KRBTEST.COM
I20260929 11:03:33.777189 17069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.767996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46074 (local address 127.26.120.252:43167)
0929 11:03:33.768167 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:33.768172 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:33.768198 (+ 26us) server_negotiation.cc:407] Connection header received
0929 11:03:33.768258 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:33.768261 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:33.768275 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:33.768336 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:33.769065 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.770181 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.771188 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.771656 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.771730 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:33.774268 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:33.774276 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:33.774276 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:33.774283 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:33.775962 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:33.776317 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:33.776320 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:33.776321 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:33.776385 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:33.776596 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:33.776597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:33.776597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:33.776739 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:33.776913 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:33.776964 (+ 51us) server_negotiation.cc:299] Negotiation successful
0929 11:03:33.777058 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":90,"threads_started":1}
I20260929 11:03:33.778090 16915 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.120.252" port: 43167 } attempt: 2
I20260929 11:03:33.778231 17067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.771214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.253:44909 (local address 127.0.0.1:40734)
0929 11:03:33.771317 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:33.771327 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:33.771368 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:33.771486 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:33.771487 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:33.771630 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:03:33.771695 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:33.771697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.772594 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.772594 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:33.773101 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:33.773102 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.773515 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.773516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:33.773595 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:33.774032 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:03:33.774038 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:03:33.775342 (+ 1304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:03:33.776921 (+ 1579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:33.776922 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:33.776923 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:33.777156 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:33.777522 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:33.777522 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:33.777522 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:33.777574 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:33.777885 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:03:33.777885 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:03:33.777946 (+ 61us) client_negotiation.cc:769] Sending connection context
0929 11:03:33.778013 (+ 67us) client_negotiation.cc:240] Negotiation successful
0929 11:03:33.778140 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260929 11:03:33.778394 16995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.771268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40734 (local address 127.26.120.253:44909)
0929 11:03:33.771320 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:33.771322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:33.771356 (+ 34us) server_negotiation.cc:407] Connection header received
0929 11:03:33.771425 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:33.771426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:33.771434 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:33.771472 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:33.771765 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.772485 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.773181 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.773435 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.773517 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:33.775418 (+ 1901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:33.775419 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:33.775419 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:33.775426 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:33.776789 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:33.777239 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:33.777240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:33.777241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:33.777313 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:33.777642 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:33.777643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:33.777643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:33.777800 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:33.777888 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:33.778122 (+ 234us) server_negotiation.cc:299] Negotiation successful
0929 11:03:33.778205 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 11:03:33.778995 17065 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.120.252" port: 43167 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 11:03:33 dist-test-slave-2lzq krb5kdc[16834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679813, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for kudu/127.26.120.252@KRBTEST.COM
Sep 29 11:03:33 dist-test-slave-2lzq krb5kdc[16834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679813, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.252@KRBTEST.COM for kudu/127.26.120.252@KRBTEST.COM
Sep 29 11:03:33 dist-test-slave-2lzq krb5kdc[16834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:03:33.786942 17067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.779230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:43167 (local address 127.0.0.1:46090)
0929 11:03:33.779271 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:33.779280 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:33.779309 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:33.779525 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:33.779526 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:33.779656 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:03:33.779707 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:33.779708 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.780317 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.780317 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:33.780960 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:33.780962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.781368 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.781369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:33.781413 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:33.781790 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:03:33.781795 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:03:33.783200 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:03:33.785882 (+ 2682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:33.785883 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:33.785883 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:33.786048 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:33.786278 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:33.786279 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:33.786279 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:33.786334 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:33.786624 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:03:33.786626 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:03:33.786666 (+ 40us) client_negotiation.cc:769] Sending connection context
0929 11:03:33.786740 (+ 74us) client_negotiation.cc:240] Negotiation successful
0929 11:03:33.786832 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260929 11:03:33.786984 16917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.778319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:43167 (local address 127.0.0.1:46082)
0929 11:03:33.778383 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:33.778394 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:33.778437 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:33.778688 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:33.778689 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:33.778851 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:03:33.778894 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:33.778896 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.779757 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.779758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:33.780517 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:33.780519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.780978 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.780979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:33.781031 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:33.781465 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:03:33.781471 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:03:33.783614 (+ 2143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:03:33.785872 (+ 2258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:33.785873 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:33.785873 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:33.786104 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:33.786329 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:33.786330 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:33.786330 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:33.786411 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:33.786681 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:03:33.786682 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:03:33.786725 (+ 43us) client_negotiation.cc:769] Sending connection context
0929 11:03:33.786791 (+ 66us) client_negotiation.cc:240] Negotiation successful
0929 11:03:33.786886 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260929 11:03:33.787051 17070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.779274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46090 (local address 127.26.120.252:43167)
0929 11:03:33.779415 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:33.779416 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:33.779422 (+ 6us) server_negotiation.cc:407] Connection header received
0929 11:03:33.779437 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:33.779438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:33.779446 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:33.779507 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:33.779780 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.780239 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.781032 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.781286 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.781505 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:33.783292 (+ 1787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:33.783293 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:33.783293 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:33.783299 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:33.785783 (+ 2484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:33.786149 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:33.786150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:33.786150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:33.786209 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:33.786387 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:33.786388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:33.786388 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:33.786550 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:33.786622 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:33.786762 (+ 140us) server_negotiation.cc:299] Negotiation successful
0929 11:03:33.786882 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":89,"server-negotiator.queue_time_us":124,"thread_start_us":89,"threads_started":1}
I20260929 11:03:33.787318 17069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.778354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46082 (local address 127.26.120.252:43167)
0929 11:03:33.778387 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:33.778389 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:33.778426 (+ 37us) server_negotiation.cc:407] Connection header received
0929 11:03:33.778603 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:33.778603 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:33.778615 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:33.778668 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:33.778968 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.779671 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.780595 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.780906 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.780976 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:33.783738 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:33.783740 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:33.783740 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:33.783747 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:33.785760 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:33.786187 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:33.786188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:33.786188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:33.786241 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:33.786461 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:33.786462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:33.786462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:33.786573 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:33.786693 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:33.787154 (+ 461us) server_negotiation.cc:299] Negotiation successful
0929 11:03:33.787234 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 11:03:33.788319 17065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3af9fff7b6949a48f898b8e799c546c: Bootstrap starting.
I20260929 11:03:33.788800 16915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e254883532840fbb608428bfeef47ef: Bootstrap starting.
I20260929 11:03:33.789003 17065 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3af9fff7b6949a48f898b8e799c546c: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:33.789278 17065 log.cc:824] T 00000000000000000000000000000000 P f3af9fff7b6949a48f898b8e799c546c: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:33.789446 16915 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e254883532840fbb608428bfeef47ef: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:33.789731 16915 log.cc:824] T 00000000000000000000000000000000 P 5e254883532840fbb608428bfeef47ef: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:33.789803 16918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.778103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57918 (local address 127.26.120.254:33691)
0929 11:03:33.778143 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:33.778145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:33.778544 (+ 399us) server_negotiation.cc:407] Connection header received
0929 11:03:33.778614 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:33.778615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:33.778624 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:33.778675 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:33.779302 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.779942 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.780839 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.781102 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.781791 (+ 689us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:33.784127 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:33.784129 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:33.784129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:33.784136 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:33.785919 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:33.786341 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:33.786342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:33.786342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:33.786429 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:33.786632 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:33.786633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:33.786633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:33.786743 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:33.786921 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:33.789571 (+ 2650us) server_negotiation.cc:299] Negotiation successful
0929 11:03:33.789654 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 11:03:33.790247 17065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3af9fff7b6949a48f898b8e799c546c: No bootstrap required, opened a new log
I20260929 11:03:33.790558 16915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e254883532840fbb608428bfeef47ef: No bootstrap required, opened a new log
I20260929 11:03:33.791692 17065 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3af9fff7b6949a48f898b8e799c546c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e254883532840fbb608428bfeef47ef" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 33691 } } peers { permanent_uuid: "e0c862ca18a84ef8a8895b2334a74692" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 44909 } } peers { permanent_uuid: "f3af9fff7b6949a48f898b8e799c546c" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 43167 } }
I20260929 11:03:33.791838 17065 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3af9fff7b6949a48f898b8e799c546c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3af9fff7b6949a48f898b8e799c546c, State: Initialized, Role: FOLLOWER
I20260929 11:03:33.791932 17065 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3af9fff7b6949a48f898b8e799c546c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e254883532840fbb608428bfeef47ef" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 33691 } } peers { permanent_uuid: "e0c862ca18a84ef8a8895b2334a74692" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 44909 } } peers { permanent_uuid: "f3af9fff7b6949a48f898b8e799c546c" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 43167 } }
I20260929 11:03:33.792019 16915 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e254883532840fbb608428bfeef47ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e254883532840fbb608428bfeef47ef" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 33691 } } peers { permanent_uuid: "e0c862ca18a84ef8a8895b2334a74692" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 44909 } } peers { permanent_uuid: "f3af9fff7b6949a48f898b8e799c546c" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 43167 } }
I20260929 11:03:33.792189 16915 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e254883532840fbb608428bfeef47ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e254883532840fbb608428bfeef47ef, State: Initialized, Role: FOLLOWER
I20260929 11:03:33.792202 17073 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3af9fff7b6949a48f898b8e799c546c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e254883532840fbb608428bfeef47ef" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 33691 } } peers { permanent_uuid: "e0c862ca18a84ef8a8895b2334a74692" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 44909 } } peers { permanent_uuid: "f3af9fff7b6949a48f898b8e799c546c" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 43167 } } }
I20260929 11:03:33.792258 17065 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3af9fff7b6949a48f898b8e799c546c [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:03:33.792305 17073 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3af9fff7b6949a48f898b8e799c546c [sys.catalog]: This master's current role is: FOLLOWER
I20260929 11:03:33.792281 16915 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e254883532840fbb608428bfeef47ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e254883532840fbb608428bfeef47ef" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 33691 } } peers { permanent_uuid: "e0c862ca18a84ef8a8895b2334a74692" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 44909 } } peers { permanent_uuid: "f3af9fff7b6949a48f898b8e799c546c" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 43167 } }
I20260929 11:03:33.792528 17074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e254883532840fbb608428bfeef47ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e254883532840fbb608428bfeef47ef" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 33691 } } peers { permanent_uuid: "e0c862ca18a84ef8a8895b2334a74692" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 44909 } } peers { permanent_uuid: "f3af9fff7b6949a48f898b8e799c546c" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 43167 } } }
I20260929 11:03:33.792585 16915 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e254883532840fbb608428bfeef47ef [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:03:33.792610 17074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e254883532840fbb608428bfeef47ef [sys.catalog]: This master's current role is: FOLLOWER
W20260929 11:03:33.794538 17092 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f3af9fff7b6949a48f898b8e799c546c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 11:03:33.794636 17092 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260929 11:03:33.795188 17096 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5e254883532840fbb608428bfeef47ef: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 11:03:33.795300 17096 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 11:03:33.795728 16997 master_runner.cc:186] Error getting master registration for 127.26.120.253:44909: OK, instance_id { permanent_uuid: "e0c862ca18a84ef8a8895b2334a74692" instance_seqno: 1790679813601902 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 11:03:33.798153 16997 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 11:03:33.795750 16847 master_runner.cc:186] Error getting master registration for 127.26.120.253:44909: OK, instance_id { permanent_uuid: "e0c862ca18a84ef8a8895b2334a74692" instance_seqno: 1790679813601902 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 11:03:33.798702 16847 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Sep 29 11:03:33 dist-test-slave-2lzq krb5kdc[16834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.253@KRBTEST.COM
I20260929 11:03:33.807780 16995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.794592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40742 (local address 127.26.120.253:44909)
0929 11:03:33.798800 (+ 4208us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:33.798803 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:33.798813 (+ 10us) server_negotiation.cc:407] Connection header received
0929 11:03:33.798838 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:33.798839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:33.798851 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:33.798920 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:33.799566 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.800308 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.801336 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.801672 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.801750 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:33.804651 (+ 2901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:33.804652 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:33.804653 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:33.804660 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:33.806471 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:33.806877 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:33.806879 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:33.806879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:33.806941 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:33.807155 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:33.807156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:33.807156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:33.807297 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:33.807461 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:33.807504 (+ 43us) server_negotiation.cc:299] Negotiation successful
0929 11:03:33.807579 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4181}
I20260929 11:03:33.809212 27107 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 11:03:33.810220 16889 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "5e254883532840fbb608428bfeef47ef"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57918
I20260929 11:03:33.810367 16889 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e254883532840fbb608428bfeef47ef [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260929 11:03:33.810412 16889 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e254883532840fbb608428bfeef47ef [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:33.811322 16889 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e254883532840fbb608428bfeef47ef [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e254883532840fbb608428bfeef47ef" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 33691 } } peers { permanent_uuid: "e0c862ca18a84ef8a8895b2334a74692" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 44909 } } peers { permanent_uuid: "f3af9fff7b6949a48f898b8e799c546c" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 43167 } }
I20260929 11:03:33.811542 16889 leader_election.cc:288] T 00000000000000000000000000000000 P 5e254883532840fbb608428bfeef47ef [CANDIDATE]: Term 1 election: Requested vote from peers e0c862ca18a84ef8a8895b2334a74692 (127.26.120.253:44909), f3af9fff7b6949a48f898b8e799c546c (127.26.120.252:43167)
I20260929 11:03:33.812012 17039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5e254883532840fbb608428bfeef47ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f3af9fff7b6949a48f898b8e799c546c"
I20260929 11:03:33.812134 17039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3af9fff7b6949a48f898b8e799c546c [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:33.812989 17039 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f3af9fff7b6949a48f898b8e799c546c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e254883532840fbb608428bfeef47ef in term 1.
I20260929 11:03:33.813280 16866 leader_election.cc:302] T 00000000000000000000000000000000 P 5e254883532840fbb608428bfeef47ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e254883532840fbb608428bfeef47ef, f3af9fff7b6949a48f898b8e799c546c; no voters:
I20260929 11:03:33.813402 17074 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e254883532840fbb608428bfeef47ef [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:03:33.813567 17074 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e254883532840fbb608428bfeef47ef [term 1 LEADER]: Becoming Leader. State: Replica: 5e254883532840fbb608428bfeef47ef, State: Running, Role: LEADER
I20260929 11:03:33.813655 17074 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e254883532840fbb608428bfeef47ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e254883532840fbb608428bfeef47ef" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 33691 } } peers { permanent_uuid: "e0c862ca18a84ef8a8895b2334a74692" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 44909 } } peers { permanent_uuid: "f3af9fff7b6949a48f898b8e799c546c" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 43167 } }
I20260929 11:03:33.814092 17104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e254883532840fbb608428bfeef47ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e254883532840fbb608428bfeef47ef. Latest consensus state: current_term: 1 leader_uuid: "5e254883532840fbb608428bfeef47ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e254883532840fbb608428bfeef47ef" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 33691 } } peers { permanent_uuid: "e0c862ca18a84ef8a8895b2334a74692" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 44909 } } peers { permanent_uuid: "f3af9fff7b6949a48f898b8e799c546c" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 43167 } } }
I20260929 11:03:33.814221 17104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e254883532840fbb608428bfeef47ef [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:33.822525 17069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.813226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46106 (local address 127.26.120.252:43167)
0929 11:03:33.813264 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:33.813267 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:33.814218 (+ 951us) server_negotiation.cc:407] Connection header received
0929 11:03:33.814313 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:33.814314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:33.814324 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:33.814354 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:33.815140 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.815894 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.816951 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.817269 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.817312 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:33.818912 (+ 1600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:33.818913 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:33.818914 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:33.818921 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:33.820652 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:33.821099 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:33.821100 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:33.821101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:33.821424 (+ 323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:33.821644 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:33.821645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:33.821645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:33.822018 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:33.822117 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:33.822334 (+ 217us) server_negotiation.cc:299] Negotiation successful
0929 11:03:33.822429 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 11:03:33.824251 16918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.813635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57934 (local address 127.26.120.254:33691)
0929 11:03:33.813671 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:33.813672 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:33.814535 (+ 863us) server_negotiation.cc:407] Connection header received
0929 11:03:33.814693 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:33.814694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:33.814704 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:33.814734 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:33.815842 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.816588 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.818230 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.818519 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.818565 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:33.820916 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:33.820917 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:33.820917 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:33.820924 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:33.822605 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:33.823208 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:33.823209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:33.823209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:33.823277 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:33.823568 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:33.823569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:33.823569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:33.823714 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:33.823781 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:33.824085 (+ 304us) server_negotiation.cc:299] Negotiation successful
0929 11:03:33.824161 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 11:03:33.830274 17108 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:03:33.830756 17108 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:03:33.832413 17039 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f3af9fff7b6949a48f898b8e799c546c [term 1 FOLLOWER]: Refusing update from remote peer 5e254883532840fbb608428bfeef47ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 11:03:33.832789 17104 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5e254883532840fbb608428bfeef47ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3af9fff7b6949a48f898b8e799c546c" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 43167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 11:03:33.835692 17104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e254883532840fbb608428bfeef47ef [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5e254883532840fbb608428bfeef47ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e254883532840fbb608428bfeef47ef" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 33691 } } peers { permanent_uuid: "e0c862ca18a84ef8a8895b2334a74692" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 44909 } } peers { permanent_uuid: "f3af9fff7b6949a48f898b8e799c546c" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 43167 } } }
I20260929 11:03:33.835776 17104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e254883532840fbb608428bfeef47ef [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:33.835742 17073 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3af9fff7b6949a48f898b8e799c546c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e254883532840fbb608428bfeef47ef. Latest consensus state: current_term: 1 leader_uuid: "5e254883532840fbb608428bfeef47ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e254883532840fbb608428bfeef47ef" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 33691 } } peers { permanent_uuid: "e0c862ca18a84ef8a8895b2334a74692" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 44909 } } peers { permanent_uuid: "f3af9fff7b6949a48f898b8e799c546c" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 43167 } } }
I20260929 11:03:33.835855 17073 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3af9fff7b6949a48f898b8e799c546c [sys.catalog]: This master's current role is: FOLLOWER
I20260929 11:03:33.836162 17109 mvcc.cc:205] Tried to move back new op lower bound from 7334624517454921728 to 7334624517381226496. Current Snapshot: MvccSnapshot[applied={T|T < 7334624517454921728}]
I20260929 11:03:33.836408 17073 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3af9fff7b6949a48f898b8e799c546c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5e254883532840fbb608428bfeef47ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e254883532840fbb608428bfeef47ef" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 33691 } } peers { permanent_uuid: "e0c862ca18a84ef8a8895b2334a74692" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 44909 } } peers { permanent_uuid: "f3af9fff7b6949a48f898b8e799c546c" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 43167 } } }
I20260929 11:03:33.836509 17073 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3af9fff7b6949a48f898b8e799c546c [sys.catalog]: This master's current role is: FOLLOWER
I20260929 11:03:33.836871 17108 catalog_manager.cc:1347] Generated new cluster ID: 0d9627937f964738af01f7f45b6a4133
I20260929 11:03:33.836889 16995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.813954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40756 (local address 127.26.120.253:44909)
0929 11:03:33.813988 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:33.813990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:33.818678 (+ 4688us) server_negotiation.cc:407] Connection header received
0929 11:03:33.818776 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:33.818778 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:33.818789 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:33.818819 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:33.820213 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.820937 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.830932 (+ 9995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.831242 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.831321 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:33.832804 (+ 1483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:33.832805 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:33.832806 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:33.832825 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:33.835363 (+ 2538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:33.835785 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:33.835786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:33.835787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:33.835846 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:33.836060 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:33.836061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:33.836062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:33.836214 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:33.836313 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:33.836712 (+ 399us) server_negotiation.cc:299] Negotiation successful
0929 11:03:33.836799 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 11:03:33.836915 17108 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 11:03:33.837250 17110 mvcc.cc:205] Tried to move back new op lower bound from 7334624517454921728 to 7334624517381226496. Current Snapshot: MvccSnapshot[applied={T|T < 7334624517454921728}]
I20260929 11:03:33.840930 16990 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.120.252" port: 43167 } attempt: 3
Sep 29 11:03:33 dist-test-slave-2lzq krb5kdc[16834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679813, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.253@KRBTEST.COM for kudu/127.26.120.252@KRBTEST.COM
I20260929 11:03:33.850841 16993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.841161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:43167 (local address 127.0.0.1:46112)
0929 11:03:33.841214 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:33.841223 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:33.841254 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:33.841417 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:33.841418 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:33.841564 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:03:33.841610 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:33.841612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.843147 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.843148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:33.843962 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:33.843965 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.844514 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.844515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:33.844583 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:33.845104 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:03:33.845109 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:03:33.847508 (+ 2399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:03:33.849609 (+ 2101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:33.849610 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:33.849610 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:33.849827 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:33.850100 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:33.850101 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:33.850101 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:33.850175 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:33.850511 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:03:33.850512 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:03:33.850569 (+ 57us) client_negotiation.cc:769] Sending connection context
0929 11:03:33.850643 (+ 74us) client_negotiation.cc:240] Negotiation successful
0929 11:03:33.850738 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260929 11:03:33.852180 17069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:33.841213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46112 (local address 127.26.120.252:43167)
0929 11:03:33.841250 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:33.841252 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:33.841261 (+ 9us) server_negotiation.cc:407] Connection header received
0929 11:03:33.841296 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:33.841297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:33.841307 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:33.841369 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:33.841693 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.842475 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.844072 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:33.844391 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:33.844449 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:33.847608 (+ 3159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:33.847609 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:33.847609 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:33.847617 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:33.849490 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:33.849918 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:33.849920 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:33.849920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:33.849990 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:33.850253 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:33.850254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:33.850254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:33.850407 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:33.850467 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:33.852010 (+ 1543us) server_negotiation.cc:299] Negotiation successful
0929 11:03:33.852090 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 11:03:33.854022 16990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0c862ca18a84ef8a8895b2334a74692: Bootstrap starting.
I20260929 11:03:33.854775 16990 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0c862ca18a84ef8a8895b2334a74692: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:33.855063 16990 log.cc:824] T 00000000000000000000000000000000 P e0c862ca18a84ef8a8895b2334a74692: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:33.856135 16990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0c862ca18a84ef8a8895b2334a74692: No bootstrap required, opened a new log
I20260929 11:03:33.859313 16990 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0c862ca18a84ef8a8895b2334a74692 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e254883532840fbb608428bfeef47ef" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 33691 } } peers { permanent_uuid: "e0c862ca18a84ef8a8895b2334a74692" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 44909 } } peers { permanent_uuid: "f3af9fff7b6949a48f898b8e799c546c" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 43167 } }
I20260929 11:03:33.859459 16990 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0c862ca18a84ef8a8895b2334a74692 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0c862ca18a84ef8a8895b2334a74692, State: Initialized, Role: FOLLOWER
I20260929 11:03:33.859534 16990 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0c862ca18a84ef8a8895b2334a74692 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e254883532840fbb608428bfeef47ef" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 33691 } } peers { permanent_uuid: "e0c862ca18a84ef8a8895b2334a74692" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 44909 } } peers { permanent_uuid: "f3af9fff7b6949a48f898b8e799c546c" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 43167 } }
I20260929 11:03:33.859844 16990 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0c862ca18a84ef8a8895b2334a74692 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:03:33.862026 16964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5e254883532840fbb608428bfeef47ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e0c862ca18a84ef8a8895b2334a74692"
I20260929 11:03:33.862156 16964 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0c862ca18a84ef8a8895b2334a74692 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:33.862965 16964 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e0c862ca18a84ef8a8895b2334a74692 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e254883532840fbb608428bfeef47ef in term 1.
I20260929 11:03:33.863170 16963 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e0c862ca18a84ef8a8895b2334a74692 [term 1 FOLLOWER]: Refusing update from remote peer 5e254883532840fbb608428bfeef47ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 11:03:33.863575 17074 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5e254883532840fbb608428bfeef47ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0c862ca18a84ef8a8895b2334a74692" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 44909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 11:03:33.864167 17108 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 11:03:33.864898 17108 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:03:33.867604 17130 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0c862ca18a84ef8a8895b2334a74692 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e254883532840fbb608428bfeef47ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e254883532840fbb608428bfeef47ef" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 33691 } } peers { permanent_uuid: "e0c862ca18a84ef8a8895b2334a74692" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 44909 } } peers { permanent_uuid: "f3af9fff7b6949a48f898b8e799c546c" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 43167 } } }
I20260929 11:03:33.867758 17130 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0c862ca18a84ef8a8895b2334a74692 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 11:03:33.867841 17130 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0c862ca18a84ef8a8895b2334a74692 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e254883532840fbb608428bfeef47ef. Latest consensus state: current_term: 1 leader_uuid: "5e254883532840fbb608428bfeef47ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e254883532840fbb608428bfeef47ef" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 33691 } } peers { permanent_uuid: "e0c862ca18a84ef8a8895b2334a74692" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 44909 } } peers { permanent_uuid: "f3af9fff7b6949a48f898b8e799c546c" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 43167 } } }
I20260929 11:03:33.867934 17130 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0c862ca18a84ef8a8895b2334a74692 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 11:03:33.868737 17104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e254883532840fbb608428bfeef47ef [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5e254883532840fbb608428bfeef47ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e254883532840fbb608428bfeef47ef" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 33691 } } peers { permanent_uuid: "e0c862ca18a84ef8a8895b2334a74692" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 44909 } } peers { permanent_uuid: "f3af9fff7b6949a48f898b8e799c546c" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 43167 } } }
I20260929 11:03:33.868821 17104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e254883532840fbb608428bfeef47ef [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:33.868355 17128 mvcc.cc:205] Tried to move back new op lower bound from 7334624517454921728 to 7334624517381226496. Current Snapshot: MvccSnapshot[applied={T|T < 7334624517454921728}]
I20260929 11:03:33.869506 17130 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0c862ca18a84ef8a8895b2334a74692 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5e254883532840fbb608428bfeef47ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e254883532840fbb608428bfeef47ef" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 33691 } } peers { permanent_uuid: "e0c862ca18a84ef8a8895b2334a74692" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 44909 } } peers { permanent_uuid: "f3af9fff7b6949a48f898b8e799c546c" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 43167 } } }
I20260929 11:03:33.869634 17130 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0c862ca18a84ef8a8895b2334a74692 [sys.catalog]: This master's current role is: FOLLOWER
W20260929 11:03:33.870561 17126 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e0c862ca18a84ef8a8895b2334a74692: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 11:03:33.870657 17126 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 11:03:33.871814 17108 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e254883532840fbb608428bfeef47ef: Generated new TSK 0
I20260929 11:03:33.871980 17108 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 11:03:33.897902 17069 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 11:03:33.896489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46128 (local address 127.26.120.252:43167)
0929 11:03:33.896534 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:33.896537 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:33.897222 (+ 685us) server_negotiation.cc:407] Connection header received
0929 11:03:33.897330 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:33.897331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:33.897342 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:33.897374 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:33.897836 (+ 462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46128: BlockingRecv error: recv got EOF from 127.0.0.1:46128 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260929 11:03:33.898852 16918 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 11:03:33.896903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57944 (local address 127.26.120.254:33691)
0929 11:03:33.896943 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:33.896945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:33.898347 (+ 1402us) server_negotiation.cc:407] Connection header received
0929 11:03:33.898444 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:33.898445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:33.898455 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:33.898484 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:33.898792 (+ 308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57944: BlockingRecv error: recv got EOF from 127.0.0.1:57944 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260929 11:03:33.900028 16995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 11:03:33.899339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40772 (local address 127.26.120.253:44909)
0929 11:03:33.899379 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:33.899381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:33.899499 (+ 118us) server_negotiation.cc:407] Connection header received
0929 11:03:33.899591 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:33.899592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:33.899602 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:33.899630 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:33.899955 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40772: BlockingRecv error: recv got EOF from 127.0.0.1:40772 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260929 11:03:33.902436 16995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 11:03:33.901644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40778 (local address 127.26.120.253:44909)
0929 11:03:33.901677 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:33.901679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:33.901863 (+ 184us) server_negotiation.cc:407] Connection header received
0929 11:03:33.901968 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:33.901969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:33.901985 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:33.902015 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:33.902397 (+ 382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40778: BlockingRecv error: recv got EOF from 127.0.0.1:40778 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260929 11:03:33.905491 16995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 11:03:33.904764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40786 (local address 127.26.120.253:44909)
0929 11:03:33.904809 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:33.904811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:33.905015 (+ 204us) server_negotiation.cc:407] Connection header received
0929 11:03:33.905113 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:33.905113 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:33.905123 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:33.905150 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:33.905450 (+ 300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40786: BlockingRecv error: recv got EOF from 127.0.0.1:40786 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260929 11:03:33.909425 16995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 11:03:33.908689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40802 (local address 127.26.120.253:44909)
0929 11:03:33.908722 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:33.908723 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:33.908939 (+ 216us) server_negotiation.cc:407] Connection header received
0929 11:03:33.909034 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:33.909034 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:33.909044 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:33.909072 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:33.909387 (+ 315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40802: BlockingRecv error: recv got EOF from 127.0.0.1:40802 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260929 11:03:33.915350 16995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 11:03:33.914613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40812 (local address 127.26.120.253:44909)
0929 11:03:33.914646 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:33.914648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:33.914864 (+ 216us) server_negotiation.cc:407] Connection header received
0929 11:03:33.914967 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:33.914968 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:33.914979 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:33.915007 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:33.915311 (+ 304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40812: BlockingRecv error: recv got EOF from 127.0.0.1:40812 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260929 11:03:33.925419 16995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 11:03:33.924621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40818 (local address 127.26.120.253:44909)
0929 11:03:33.924658 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:33.924660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:33.924892 (+ 232us) server_negotiation.cc:407] Connection header received
0929 11:03:33.924991 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:33.924992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:33.925002 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:33.925031 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:33.925361 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40818: BlockingRecv error: recv got EOF from 127.0.0.1:40818 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260929 11:03:33.943689 16995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 11:03:33.942870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40824 (local address 127.26.120.253:44909)
0929 11:03:33.942911 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:33.942914 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:33.943153 (+ 239us) server_negotiation.cc:407] Connection header received
0929 11:03:33.943248 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:33.943250 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:33.943261 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:33.943291 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:33.943635 (+ 344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40824: BlockingRecv error: recv got EOF from 127.0.0.1:40824 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260929 11:03:33.977878 16995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 11:03:33.977092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40840 (local address 127.26.120.253:44909)
0929 11:03:33.977148 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:33.977150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:33.977300 (+ 150us) server_negotiation.cc:407] Connection header received
0929 11:03:33.977377 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:33.977378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:33.977388 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:33.977461 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:33.977806 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40840: BlockingRecv error: recv got EOF from 127.0.0.1:40840 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260929 11:03:34.043777 16995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 11:03:34.043123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40856 (local address 127.26.120.253:44909)
0929 11:03:34.043194 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:34.043197 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:34.043295 (+ 98us) server_negotiation.cc:407] Connection header received
0929 11:03:34.043329 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:34.043330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:34.043340 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:34.043408 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:34.043706 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40856: BlockingRecv error: recv got EOF from 127.0.0.1:40856 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260929 11:03:34.173900 16995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 11:03:34.173134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40866 (local address 127.26.120.253:44909)
0929 11:03:34.173231 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:34.173233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:34.173247 (+ 14us) server_negotiation.cc:407] Connection header received
0929 11:03:34.173351 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:34.173352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:34.173366 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:34.173480 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:34.173827 (+ 347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40866: BlockingRecv error: recv got EOF from 127.0.0.1:40866 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260929 11:03:34.431774 16995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 11:03:34.431132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40878 (local address 127.26.120.253:44909)
0929 11:03:34.431225 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:34.431227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:34.431256 (+ 29us) server_negotiation.cc:407] Connection header received
0929 11:03:34.431286 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:34.431287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:34.431302 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:34.431370 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:34.431727 (+ 357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40878: BlockingRecv error: recv got EOF from 127.0.0.1:40878 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 11:03:34.795094 17092 catalog_manager.cc:1259] Loaded cluster ID: 0d9627937f964738af01f7f45b6a4133
I20260929 11:03:34.795169 17092 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f3af9fff7b6949a48f898b8e799c546c: loading cluster ID for follower catalog manager: success
I20260929 11:03:34.795964 17092 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f3af9fff7b6949a48f898b8e799c546c: acquiring CA information for follower catalog manager: success
I20260929 11:03:34.796203 17092 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f3af9fff7b6949a48f898b8e799c546c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 11:03:34.871161 17126 catalog_manager.cc:1259] Loaded cluster ID: 0d9627937f964738af01f7f45b6a4133
I20260929 11:03:34.871269 17126 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e0c862ca18a84ef8a8895b2334a74692: loading cluster ID for follower catalog manager: success
I20260929 11:03:34.872094 17126 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e0c862ca18a84ef8a8895b2334a74692: acquiring CA information for follower catalog manager: success
I20260929 11:03:34.872423 17126 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e0c862ca18a84ef8a8895b2334a74692: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 11:03:34.947540 17198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:34.945071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40884 (local address 127.26.120.253:44909)
0929 11:03:34.945199 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:34.945201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:34.945211 (+ 10us) server_negotiation.cc:407] Connection header received
0929 11:03:34.945258 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:34.945259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:34.945307 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:34.945386 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 11:03:34.945530 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:34.946047 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:34.946778 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:34.946982 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:34.947037 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:34.947179 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 11:03:34.947285 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 11:03:34.947341 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:34.947386 (+ 45us) server_negotiation.cc:299] Negotiation successful
0929 11:03:34.947403 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":62,"threads_started":1}
I20260929 11:03:34.951999 17204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:34.949395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46140 (local address 127.26.120.252:43167)
0929 11:03:34.949511 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:34.949513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:34.949522 (+ 9us) server_negotiation.cc:407] Connection header received
0929 11:03:34.949599 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:34.949600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:34.949645 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:34.949743 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 11:03:34.949857 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:34.950434 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:34.951180 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:34.951402 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:34.951478 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:34.951631 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 11:03:34.951745 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 11:03:34.951833 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:34.951880 (+ 47us) server_negotiation.cc:299] Negotiation successful
0929 11:03:34.951902 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":53,"threads_started":1}
I20260929 11:03:34.956317 17204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:34.953620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46154 (local address 127.26.120.252:43167)
0929 11:03:34.953658 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:34.953659 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:34.953858 (+ 199us) server_negotiation.cc:407] Connection header received
0929 11:03:34.953911 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:34.953912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:34.953922 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:34.953970 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 11:03:34.954169 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:34.954885 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:34.955624 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:34.955885 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:34.955948 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:34.956062 (+ 114us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 11:03:34.956092 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 11:03:34.956150 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:34.956225 (+ 75us) server_negotiation.cc:299] Negotiation successful
0929 11:03:34.956246 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 11:03:34.956755 17198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:34.953723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40888 (local address 127.26.120.253:44909)
0929 11:03:34.953760 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:34.953762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:34.953773 (+ 11us) server_negotiation.cc:407] Connection header received
0929 11:03:34.953801 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:34.953802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:34.953812 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:34.953876 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 11:03:34.954030 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:34.954811 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:34.955835 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:34.956131 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:34.956193 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:34.956350 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 11:03:34.956384 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 11:03:34.956432 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:34.956651 (+ 219us) server_negotiation.cc:299] Negotiation successful
0929 11:03:34.956675 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 11:03:34.957239 17211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:34.953622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57946 (local address 127.26.120.254:33691)
0929 11:03:34.953853 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:34.953855 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:34.953874 (+ 19us) server_negotiation.cc:407] Connection header received
0929 11:03:34.953920 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:34.953921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:34.953972 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:34.954218 (+ 246us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 11:03:34.954262 (+ 44us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:34.954726 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:34.956234 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:34.956541 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:34.956609 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:34.956839 (+ 230us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 11:03:34.956948 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 11:03:34.957044 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:34.957108 (+ 64us) server_negotiation.cc:299] Negotiation successful
0929 11:03:34.957157 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":57,"threads_started":1}
I20260929 11:03:34.958906 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 16847
I20260929 11:03:34.964471 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 16922
I20260929 11:03:34.969676 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 16997
2026-09-29T11:03:34Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2802 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 11:03:35 dist-test-slave-2lzq krb5kdc[17215](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 11:03:35 dist-test-slave-2lzq krb5kdc[17215](info): set up 2 sockets
Sep 29 11:03:35 dist-test-slave-2lzq krb5kdc[17215](info): commencing operation
krb5kdc: starting...
W20260929 11:03:35.994578 27107 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/krb5kdc/test-user.keytab.
Sep 29 11:03:36 dist-test-slave-2lzq krb5kdc[17215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T11:03:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T11:03:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.120.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
Entry for principal kudu/127.26.120.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab.
I20260929 11:03:36.112646 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:45803
--webserver_interface=127.26.120.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:44999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:45803,127.26.120.253:38943,127.26.120.252:37281
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 11:03:36.193297 17228 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 11:03:36.193558 17228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:36.193586 17228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:36.193631 17228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:36.194790 17228 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:03:36.194859 17228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:36.194885 17228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:36.194906 17228 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:03:36.194936 17228 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:03:36.196404 17228 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:44999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:45803,127.26.120.253:38943,127.26.120.252:37281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.254.keytab
--principal=kudu/127.26.120.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.254:45803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.120.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:36.196674 17228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:36.196902 17228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 11:03:36.201884 17228 server_base.cc:1034] running on GCE node
W20260929 11:03:36.201851 17234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:36.201817 17233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:36.201828 17236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:36.202260 17228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:36.202497 17228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:36.203642 17228 hybrid_clock.cc:648] HybridClock initialized: now 1790679816203633 us; error 31 us; skew 500 ppm
Sep 29 11:03:36 dist-test-slave-2lzq krb5kdc[17215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679816, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:03:36.210203 17228 init.cc:377] Logged in from keytab as kudu/127.26.120.254@KRBTEST.COM (short username kudu)
I20260929 11:03:36.210651 17228 webserver.cc:460] Webserver started at http://127.26.120.254:41431/ using document root <none> and password file <none>
I20260929 11:03:36.210903 17228 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:36.210950 17228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:36.211102 17228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:36.212227 17228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-0/data/instance:
uuid: "2043e40ac747409988a3a2a145b687d4"
format_stamp: "Formatted at 2026-09-29 11:03:36 on dist-test-slave-2lzq"
server_key: "30c3f24115e1f5d096f23786f491dc23"
server_key_iv: "ea62b499a102fbebcaba8437a894f9a4"
server_key_version: "encryptionkey@0"
I20260929 11:03:36.212602 17228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-0/wal/instance:
uuid: "2043e40ac747409988a3a2a145b687d4"
format_stamp: "Formatted at 2026-09-29 11:03:36 on dist-test-slave-2lzq"
server_key: "30c3f24115e1f5d096f23786f491dc23"
server_key_iv: "ea62b499a102fbebcaba8437a894f9a4"
server_key_version: "encryptionkey@0"
I20260929 11:03:36.214143 17228 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 11:03:36.214950 17243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:36.215143 17228 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 11:03:36.215242 17228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-0/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-0/wal
uuid: "2043e40ac747409988a3a2a145b687d4"
format_stamp: "Formatted at 2026-09-29 11:03:36 on dist-test-slave-2lzq"
server_key: "30c3f24115e1f5d096f23786f491dc23"
server_key_iv: "ea62b499a102fbebcaba8437a894f9a4"
server_key_version: "encryptionkey@0"
I20260929 11:03:36.215323 17228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:36.229533 17228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:36.231530 17228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:36.231691 17228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:36.235626 17228 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.254:45803
I20260929 11:03:36.235658 17295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.254:45803 every 8 connection(s)
I20260929 11:03:36.235989 17228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-0/data/info.pb
I20260929 11:03:36.236879 17296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:36.237483 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 17228
I20260929 11:03:36.237577 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-0/wal/instance
I20260929 11:03:36.237759 27107 external_mini_cluster.cc:1428] Setting key 1ae9d86b3fcbdffabcd81dacdebbf609
I20260929 11:03:36.239171 17296 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.120.254" port: 45803 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.26.120.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.253@KRBTEST.COM" created.
Sep 29 11:03:36 dist-test-slave-2lzq krb5kdc[17215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679816, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:03:36.251907 17300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.240204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40014 (local address 127.26.120.254:45803)
0929 11:03:36.240333 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:36.240337 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:36.240370 (+ 33us) server_negotiation.cc:407] Connection header received
0929 11:03:36.240891 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:36.240893 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:36.240917 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:36.241009 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:36.241612 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.242505 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.243406 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.243716 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.243853 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:36.248373 (+ 4520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:36.248382 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:36.248383 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:36.248391 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:36.250152 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:36.250649 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:36.250652 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:36.250652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:36.250731 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:36.250992 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:36.250993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:36.250993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:36.251299 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:36.251368 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:36.251628 (+ 260us) server_negotiation.cc:299] Negotiation successful
0929 11:03:36.251735 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":60,"threads_started":1}
I20260929 11:03:36.252031 17299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.239997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:45803 (local address 127.0.0.1:40014)
0929 11:03:36.240165 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:36.240200 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:36.240715 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:36.241017 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:36.241020 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:36.241309 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:03:36.241507 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:36.241512 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.242615 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.242615 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:36.243280 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:36.243282 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.243816 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.243817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:36.243882 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:36.244475 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:03:36.244483 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:03:36.248271 (+ 3788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:03:36.250259 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:36.250266 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:36.250269 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:36.250559 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:36.250796 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:36.250797 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:36.250797 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:36.250919 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:36.251381 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:03:36.251389 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:03:36.251475 (+ 86us) client_negotiation.cc:769] Sending connection context
0929 11:03:36.251592 (+ 117us) client_negotiation.cc:240] Negotiation successful
0929 11:03:36.251839 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":63,"thread_start_us":91,"threads_started":1}
I20260929 11:03:36.252770 17296 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.120.253" port: 38943 } has no permanent_uuid. Determining permanent_uuid...
I20260929 11:03:36.253156 17299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.253018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.253:38943 (local address 127.0.0.1:44492)
0929 11:03:36.253073 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:36.253125 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.253:38943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260929 11:03:36.253309 17244 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.120.253:38943: connect: Connection refused (error 111)
I20260929 11:03:36.253744 17299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.253642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.253:38943 (local address 127.0.0.1:44500)
0929 11:03:36.253690 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:36.253719 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.253:38943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260929 11:03:36.253875 17296 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.120.253:38943: Network error: Client connection negotiation failed: client connection to 127.26.120.253:38943: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.253.keytab.
Entry for principal kudu/127.26.120.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.253.keytab.
I20260929 11:03:36.264680 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.253:38943
--webserver_interface=127.26.120.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.253.keytab
--principal=kudu/127.26.120.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:44999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:45803,127.26.120.253:38943,127.26.120.252:37281
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 11:03:36.303094 17296 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.120.253" port: 38943 } attempt: 1
I20260929 11:03:36.303560 17299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.303427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.253:38943 (local address 127.0.0.1:44510)
0929 11:03:36.303499 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:36.303533 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.253:38943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260929 11:03:36.303898 17299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.303800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.253:38943 (local address 127.0.0.1:44516)
0929 11:03:36.303844 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:36.303871 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.253:38943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260929 11:03:36.303982 17296 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.120.253:38943: Network error: Client connection negotiation failed: client connection to 127.26.120.253:38943: connect: Connection refused (error 111)
W20260929 11:03:36.348021 17303 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 11:03:36.348269 17303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:36.348301 17303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:36.348372 17303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:36.349575 17303 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:03:36.349641 17303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:36.349665 17303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:36.349694 17303 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:03:36.349720 17303 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:03:36.351583 17303 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:44999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:45803,127.26.120.253:38943,127.26.120.252:37281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.253.keytab
--principal=kudu/127.26.120.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.253:38943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.120.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:36.351971 17303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:36.352237 17303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:36.358033 17308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:36.358143 17311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:36.358196 17309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:36.358966 17303 server_base.cc:1034] running on GCE node
I20260929 11:03:36.359262 17303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:36.359611 17303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:36.360769 17303 hybrid_clock.cc:648] HybridClock initialized: now 1790679816360746 us; error 39 us; skew 500 ppm
Sep 29 11:03:36 dist-test-slave-2lzq krb5kdc[17215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679816, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:03:36.363322 17303 init.cc:377] Logged in from keytab as kudu/127.26.120.253@KRBTEST.COM (short username kudu)
I20260929 11:03:36.363749 17303 webserver.cc:460] Webserver started at http://127.26.120.253:39261/ using document root <none> and password file <none>
I20260929 11:03:36.363996 17303 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:36.364066 17303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:36.364213 17303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:36.365260 17303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-1/data/instance:
uuid: "f2fb7867ab0f4521a3de5eaf96b2f19f"
format_stamp: "Formatted at 2026-09-29 11:03:36 on dist-test-slave-2lzq"
server_key: "56c6ca6ba25eaf7c124d75be230cf233"
server_key_iv: "b528bbe4c1895b4ca23a374f51420be6"
server_key_version: "encryptionkey@0"
I20260929 11:03:36.365619 17303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-1/wal/instance:
uuid: "f2fb7867ab0f4521a3de5eaf96b2f19f"
format_stamp: "Formatted at 2026-09-29 11:03:36 on dist-test-slave-2lzq"
server_key: "56c6ca6ba25eaf7c124d75be230cf233"
server_key_iv: "b528bbe4c1895b4ca23a374f51420be6"
server_key_version: "encryptionkey@0"
I20260929 11:03:36.367075 17303 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 11:03:36.367843 17318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:36.368040 17303 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 11:03:36.368139 17303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-1/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-1/wal
uuid: "f2fb7867ab0f4521a3de5eaf96b2f19f"
format_stamp: "Formatted at 2026-09-29 11:03:36 on dist-test-slave-2lzq"
server_key: "56c6ca6ba25eaf7c124d75be230cf233"
server_key_iv: "b528bbe4c1895b4ca23a374f51420be6"
server_key_version: "encryptionkey@0"
I20260929 11:03:36.368223 17303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:36.372110 17296 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.120.253" port: 38943 } attempt: 2
I20260929 11:03:36.372517 17299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.372371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.253:38943 (local address 127.0.0.1:44530)
0929 11:03:36.372434 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:36.372478 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.253:38943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260929 11:03:36.372898 17299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.372791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.253:38943 (local address 127.0.0.1:44540)
0929 11:03:36.372842 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:36.372872 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.253:38943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260929 11:03:36.373023 17296 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.120.253:38943: Network error: Client connection negotiation failed: client connection to 127.26.120.253:38943: connect: Connection refused (error 111)
I20260929 11:03:36.389034 17303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:36.391610 17303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:36.391780 17303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:36.396008 17303 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.253:38943
I20260929 11:03:36.396061 17370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.253:38943 every 8 connection(s)
I20260929 11:03:36.396445 17303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-1/data/info.pb
I20260929 11:03:36.397521 17371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:36.399493 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 17303
I20260929 11:03:36.399607 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-1/wal/instance
I20260929 11:03:36.399791 27107 external_mini_cluster.cc:1428] Setting key 7cece0418874855638675f940926d819
I20260929 11:03:36.399935 17371 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.120.254" port: 45803 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 11:03:36 dist-test-slave-2lzq krb5kdc[17215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679816, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.253@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.120.252@KRBTEST.COM; defaulting to no policy
I20260929 11:03:36.409889 17300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.400792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40024 (local address 127.26.120.254:45803)
0929 11:03:36.400839 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:36.400842 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:36.401103 (+ 261us) server_negotiation.cc:407] Connection header received
0929 11:03:36.401813 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:36.401815 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:36.401832 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:36.401907 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:36.402800 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.403291 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.404422 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.404656 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.404748 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:36.406999 (+ 2251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:36.407003 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:36.407003 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:36.407015 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:36.408457 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:36.408881 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:36.408882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:36.408883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:36.408966 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:36.409219 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:36.409220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:36.409221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:36.409339 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:36.409413 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:36.409697 (+ 284us) server_negotiation.cc:299] Negotiation successful
0929 11:03:36.409783 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 11:03:36.410367 17373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.400773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:45803 (local address 127.0.0.1:40024)
0929 11:03:36.400941 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:36.400993 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:36.401690 (+ 697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:36.401979 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:36.401983 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:36.402399 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:03:36.402696 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:36.402703 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.403437 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.403439 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:36.404326 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:36.404329 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.404758 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.404760 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:36.404895 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:36.405472 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:03:36.405480 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:03:36.406886 (+ 1406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:03:36.408561 (+ 1675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:36.408568 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:36.408571 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:36.408798 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:36.409029 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:36.409030 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:36.409030 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:36.409144 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:36.409416 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:03:36.409423 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:03:36.409565 (+ 142us) client_negotiation.cc:769] Sending connection context
0929 11:03:36.409678 (+ 113us) client_negotiation.cc:240] Negotiation successful
0929 11:03:36.409838 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":86,"threads_started":1}
I20260929 11:03:36.411048 17371 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.120.253" port: 38943 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.120.252@KRBTEST.COM" created.
Sep 29 11:03:36 dist-test-slave-2lzq krb5kdc[17215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679816, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.253@KRBTEST.COM for kudu/127.26.120.253@KRBTEST.COM
I20260929 11:03:36.418978 17373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.411342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.253:38943 (local address 127.0.0.1:44546)
0929 11:03:36.411394 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:36.411405 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:36.411447 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:36.411771 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:36.411772 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:36.411915 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:03:36.411960 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:36.411962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.412813 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.412814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:36.413490 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:36.413492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.414243 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.414244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:36.414317 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:36.414766 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:03:36.414771 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:03:36.415988 (+ 1217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:03:36.417825 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:36.417826 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:36.417826 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:36.418092 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:36.418323 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:36.418324 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:36.418324 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:36.418384 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:36.418735 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:03:36.418736 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:03:36.418772 (+ 36us) client_negotiation.cc:769] Sending connection context
0929 11:03:36.418823 (+ 51us) client_negotiation.cc:240] Negotiation successful
0929 11:03:36.418912 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260929 11:03:36.419126 17375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.411434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44546 (local address 127.26.120.253:38943)
0929 11:03:36.411609 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:36.411613 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:36.411629 (+ 16us) server_negotiation.cc:407] Connection header received
0929 11:03:36.411657 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:36.411659 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:36.411669 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:36.411771 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:36.412038 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.412709 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.413581 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.414099 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.414162 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:36.416084 (+ 1922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:36.416092 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:36.416092 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:36.416099 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:36.417724 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:36.418169 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:36.418173 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:36.418173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:36.418247 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:36.418453 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:36.418454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:36.418454 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:36.418649 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:36.418759 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:36.418859 (+ 100us) server_negotiation.cc:299] Negotiation successful
0929 11:03:36.418989 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":149,"thread_start_us":95,"threads_started":1}
I20260929 11:03:36.419834 17371 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.120.252" port: 37281 } has no permanent_uuid. Determining permanent_uuid...
I20260929 11:03:36.420192 17373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.420072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:37281 (local address 127.0.0.1:40018)
0929 11:03:36.420121 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:36.420163 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.252:37281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260929 11:03:36.420325 17322 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.120.252:37281: connect: Connection refused (error 111)
I20260929 11:03:36.420866 17373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.420761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:37281 (local address 127.0.0.1:40022)
0929 11:03:36.420811 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:36.420836 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.252:37281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260929 11:03:36.420974 17371 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.120.252:37281: Network error: Client connection negotiation failed: client connection to 127.26.120.252:37281: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.120.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.252.keytab.
Entry for principal kudu/127.26.120.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.252.keytab.
I20260929 11:03:36.429704 27107 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkJyhTP/build/release/bin/kudu
/tmp/dist-test-taskkJyhTP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.252:37281
--webserver_interface=127.26.120.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.252.keytab
--principal=kudu/127.26.120.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.120.212:44999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.120.254:45803,127.26.120.253:38943,127.26.120.252:37281
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 11:03:36.464161 17296 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.120.253" port: 38943 } attempt: 3
Sep 29 11:03:36 dist-test-slave-2lzq krb5kdc[17215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679816, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for kudu/127.26.120.253@KRBTEST.COM
I20260929 11:03:36.470145 17371 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.120.252" port: 37281 } attempt: 1
I20260929 11:03:36.470536 17373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.470412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:37281 (local address 127.0.0.1:40038)
0929 11:03:36.470471 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:36.470507 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.252:37281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260929 11:03:36.470923 17373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.470823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:37281 (local address 127.0.0.1:40048)
0929 11:03:36.470868 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:36.470897 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.252:37281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260929 11:03:36.471117 17371 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.120.252:37281: Network error: Client connection negotiation failed: client connection to 127.26.120.252:37281: connect: Connection refused (error 111)
I20260929 11:03:36.471863 17299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.464562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.253:38943 (local address 127.0.0.1:44556)
0929 11:03:36.464623 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:36.464638 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:36.464691 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:36.464921 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:36.464922 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:36.465075 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:03:36.465144 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:36.465145 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.466182 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.466183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:36.466708 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:36.466710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.467214 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.467215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:36.467269 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:36.467781 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:03:36.467786 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:03:36.468999 (+ 1213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:03:36.470714 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:36.470715 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:36.470716 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:36.470965 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:36.471189 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:36.471190 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:36.471191 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:36.471260 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:36.471533 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:03:36.471534 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:03:36.471580 (+ 46us) client_negotiation.cc:769] Sending connection context
0929 11:03:36.471671 (+ 91us) client_negotiation.cc:240] Negotiation successful
0929 11:03:36.471771 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260929 11:03:36.472012 17375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.464661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44556 (local address 127.26.120.253:38943)
0929 11:03:36.464739 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:36.464741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:36.464758 (+ 17us) server_negotiation.cc:407] Connection header received
0929 11:03:36.464794 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:36.464796 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:36.464812 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:36.464917 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:36.465242 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.466103 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.466794 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.467140 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.467211 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:36.469086 (+ 1875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:36.469087 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:36.469087 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:36.469095 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:36.470634 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:36.471042 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:36.471043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:36.471044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:36.471120 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:36.471327 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:36.471328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:36.471329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:36.471461 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:36.471542 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:36.471695 (+ 153us) server_negotiation.cc:299] Negotiation successful
0929 11:03:36.471772 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 11:03:36.472559 17296 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.120.252" port: 37281 } has no permanent_uuid. Determining permanent_uuid...
I20260929 11:03:36.472914 17299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.472802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:37281 (local address 127.0.0.1:40060)
0929 11:03:36.472849 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:36.472883 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.252:37281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260929 11:03:36.473260 17299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.473147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:37281 (local address 127.0.0.1:40062)
0929 11:03:36.473191 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:36.473213 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.252:37281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260929 11:03:36.473376 17296 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.120.252:37281: Network error: Client connection negotiation failed: client connection to 127.26.120.252:37281: connect: Connection refused (error 111)
I20260929 11:03:36.504525 17296 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.120.252" port: 37281 } attempt: 1
I20260929 11:03:36.505080 17299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.504932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:37281 (local address 127.0.0.1:40066)
0929 11:03:36.505001 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:36.505047 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.252:37281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260929 11:03:36.505568 17299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.505458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:37281 (local address 127.0.0.1:40080)
0929 11:03:36.505510 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:36.505544 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.252:37281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260929 11:03:36.505698 17296 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.120.252:37281: Network error: Client connection negotiation failed: client connection to 127.26.120.252:37281: connect: Connection refused (error 111)
W20260929 11:03:36.512274 17378 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 11:03:36.512485 17378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 11:03:36.512502 17378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 11:03:36.512562 17378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 11:03:36.513707 17378 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 11:03:36.513746 17378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 11:03:36.513761 17378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 11:03:36.513811 17378 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 11:03:36.513829 17378 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 11:03:36.515339 17378 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.120.212:44999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.120.254:45803,127.26.120.253:38943,127.26.120.252:37281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/krb5kdc/kudu_127.26.120.252.keytab
--principal=kudu/127.26.120.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.120.252:37281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.120.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 10:01:35 UTC on ccycloud.quasar-ehiyrj.root.comops.site
build id 4679902
I20260929 11:03:36.515587 17378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 11:03:36.515817 17378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 11:03:36.520768 17383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:36.520764 17384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 11:03:36.520761 17386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 11:03:36.520782 17378 server_base.cc:1034] running on GCE node
I20260929 11:03:36.521173 17378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 11:03:36.521524 17378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 11:03:36.522686 17378 hybrid_clock.cc:648] HybridClock initialized: now 1790679816522671 us; error 26 us; skew 500 ppm
Sep 29 11:03:36 dist-test-slave-2lzq krb5kdc[17215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679816, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 11:03:36.525027 17378 init.cc:377] Logged in from keytab as kudu/127.26.120.252@KRBTEST.COM (short username kudu)
I20260929 11:03:36.525492 17378 webserver.cc:460] Webserver started at http://127.26.120.252:33239/ using document root <none> and password file <none>
I20260929 11:03:36.525712 17378 fs_manager.cc:359] Metadata directory not provided
I20260929 11:03:36.525782 17378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 11:03:36.525941 17378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 11:03:36.527014 17378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-2/data/instance:
uuid: "c4b01f92261a492494d431a8e640d766"
format_stamp: "Formatted at 2026-09-29 11:03:36 on dist-test-slave-2lzq"
server_key: "56900e09736e2a22011d19ac15db124f"
server_key_iv: "5b9917077c5d2fdedc503e3cd1f2a446"
server_key_version: "encryptionkey@0"
I20260929 11:03:36.527376 17378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-2/wal/instance:
uuid: "c4b01f92261a492494d431a8e640d766"
format_stamp: "Formatted at 2026-09-29 11:03:36 on dist-test-slave-2lzq"
server_key: "56900e09736e2a22011d19ac15db124f"
server_key_iv: "5b9917077c5d2fdedc503e3cd1f2a446"
server_key_version: "encryptionkey@0"
I20260929 11:03:36.528687 17378 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 11:03:36.529397 17393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 11:03:36.529570 17378 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 11:03:36.529666 17378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-2/data,/tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-2/wal
uuid: "c4b01f92261a492494d431a8e640d766"
format_stamp: "Formatted at 2026-09-29 11:03:36 on dist-test-slave-2lzq"
server_key: "56900e09736e2a22011d19ac15db124f"
server_key_iv: "5b9917077c5d2fdedc503e3cd1f2a446"
server_key_version: "encryptionkey@0"
I20260929 11:03:36.529752 17378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 11:03:36.539274 17371 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.120.252" port: 37281 } attempt: 2
I20260929 11:03:36.539757 17373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.539609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:37281 (local address 127.0.0.1:40094)
0929 11:03:36.539685 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:36.539728 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.252:37281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260929 11:03:36.540109 17373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.540030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:37281 (local address 127.0.0.1:40100)
0929 11:03:36.540070 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:36.540093 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.120.252:37281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260929 11:03:36.540220 17371 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.120.252:37281: Network error: Client connection negotiation failed: client connection to 127.26.120.252:37281: connect: Connection refused (error 111)
I20260929 11:03:36.547281 17378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 11:03:36.549149 17378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 11:03:36.549310 17378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 11:03:36.553471 17378 rpc_server.cc:307] RPC server started. Bound to: 127.26.120.252:37281
I20260929 11:03:36.553503 17445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.120.252:37281 every 8 connection(s)
I20260929 11:03:36.553871 17378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-2/data/info.pb
I20260929 11:03:36.554539 27107 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkJyhTP/build/release/bin/kudu as pid 17378
I20260929 11:03:36.554649 27107 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkJyhTP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790679549986039-27107-0/minicluster-data/master-2/wal/instance
I20260929 11:03:36.554874 27107 external_mini_cluster.cc:1428] Setting key 7cba2423594400082b3733863ff13865
I20260929 11:03:36.554991 17446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 11:03:36.557525 17446 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.120.254" port: 45803 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 11:03:36 dist-test-slave-2lzq krb5kdc[17215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.252@KRBTEST.COM
Sep 29 11:03:36 dist-test-slave-2lzq krb5kdc[17215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679816, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.252@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:03:36.565699 17449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.555418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40112 (local address 127.26.120.252:37281)
0929 11:03:36.555587 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:36.555603 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:36.555628 (+ 25us) server_negotiation.cc:407] Connection header received
0929 11:03:36.556163 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:36.556171 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:36.556226 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:36.556340 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:36.557198 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.558332 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.559113 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.559534 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.559653 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:36.562398 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:36.562404 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:36.562405 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:36.562411 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:36.564049 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:36.564411 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:36.564415 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:36.564415 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:36.564514 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:36.564741 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:36.564742 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:36.564743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:36.565066 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:36.565285 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:36.565359 (+ 74us) server_negotiation.cc:299] Negotiation successful
0929 11:03:36.565490 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":83,"threads_started":1}
I20260929 11:03:36.566792 17450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.558209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.254:45803 (local address 127.0.0.1:40030)
0929 11:03:36.558407 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:36.558446 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:36.558533 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:36.558678 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:36.558680 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:36.559189 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:03:36.559272 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:36.559274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.560094 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.560095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:36.561082 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:36.561085 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.561534 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.561535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:36.561592 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:36.562129 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:03:36.562137 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:03:36.563666 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:03:36.565545 (+ 1879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:36.565551 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:36.565553 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:36.565773 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:36.566026 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:36.566027 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:36.566028 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:36.566115 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:36.566410 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:03:36.566417 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:03:36.566472 (+ 55us) client_negotiation.cc:769] Sending connection context
0929 11:03:36.566593 (+ 121us) client_negotiation.cc:240] Negotiation successful
0929 11:03:36.566708 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":92,"threads_started":1}
I20260929 11:03:36.566838 17300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.558264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40030 (local address 127.26.120.254:45803)
0929 11:03:36.558337 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:36.558339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:36.558516 (+ 177us) server_negotiation.cc:407] Connection header received
0929 11:03:36.558587 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:36.558588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:36.558598 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:36.558657 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:36.559411 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.559946 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.561178 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.561445 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.561520 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:36.563756 (+ 2236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:36.563758 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:36.563758 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:36.563765 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:36.565459 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:36.565903 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:36.565904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:36.565904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:36.565966 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:36.566183 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:36.566185 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:36.566185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:36.566322 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:36.566392 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:36.566625 (+ 233us) server_negotiation.cc:299] Negotiation successful
0929 11:03:36.566719 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 11:03:36.567288 17446 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.120.253" port: 38943 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 11:03:36 dist-test-slave-2lzq krb5kdc[17215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679816, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.252@KRBTEST.COM for kudu/127.26.120.253@KRBTEST.COM
Sep 29 11:03:36 dist-test-slave-2lzq krb5kdc[17215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.254@KRBTEST.COM
I20260929 11:03:36.574811 17450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.567521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.253:38943 (local address 127.0.0.1:44568)
0929 11:03:36.567641 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:36.567651 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:36.567701 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:36.567821 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:36.567821 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:36.567969 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:03:36.568017 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:36.568019 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.568644 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.568645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:36.569399 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:36.569401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.569823 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.569824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:36.569880 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:36.570334 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:03:36.570339 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:03:36.572084 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:03:36.573791 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:36.573792 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:36.573793 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:36.573960 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:36.574188 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:36.574189 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:36.574189 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:36.574246 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:36.574506 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:03:36.574507 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:03:36.574547 (+ 40us) client_negotiation.cc:769] Sending connection context
0929 11:03:36.574625 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 11:03:36.574719 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105}
I20260929 11:03:36.574815 17375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.567593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44568 (local address 127.26.120.253:38943)
0929 11:03:36.567633 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:36.567635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:36.567688 (+ 53us) server_negotiation.cc:407] Connection header received
0929 11:03:36.567764 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:36.567765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:36.567772 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:36.567816 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:36.568078 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.568555 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.569483 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.569747 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.569812 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:36.572159 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:36.572160 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:36.572160 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:36.572185 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:36.573688 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:36.574036 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:36.574037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:36.574038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:36.574112 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:36.574319 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:36.574320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:36.574321 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:36.574434 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:36.574502 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:36.574662 (+ 160us) server_negotiation.cc:299] Negotiation successful
0929 11:03:36.574730 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 11:03:36.575038 17451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.566562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40044 (local address 127.26.120.254:45803)
0929 11:03:36.566727 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:36.566729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:36.566736 (+ 7us) server_negotiation.cc:407] Connection header received
0929 11:03:36.566754 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:36.566755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:36.566776 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:36.566845 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:36.568010 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.568776 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.569668 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.569961 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.570039 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:36.572350 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:36.572351 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:36.572352 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:36.572358 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:36.573822 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:36.574215 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:36.574216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:36.574216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:36.574269 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:36.574474 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:36.574475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:36.574475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:36.574586 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:36.574756 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:36.574817 (+ 61us) server_negotiation.cc:299] Negotiation successful
0929 11:03:36.574894 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":83,"threads_started":1}
I20260929 11:03:36.575670 17446 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.120.252" port: 37281 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 11:03:36 dist-test-slave-2lzq krb5kdc[17215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679816, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.252@KRBTEST.COM for kudu/127.26.120.252@KRBTEST.COM
Sep 29 11:03:36 dist-test-slave-2lzq krb5kdc[17215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.120.253@KRBTEST.COM
I20260929 11:03:36.580880 17296 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.120.252" port: 37281 } attempt: 2
I20260929 11:03:36.582741 17450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.575883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:37281 (local address 127.0.0.1:40124)
0929 11:03:36.575931 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:36.575940 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:36.575974 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:36.576091 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:36.576092 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:36.576201 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:03:36.576228 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:36.576229 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.576918 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.576918 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:36.577483 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:36.577485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.577892 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.577893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:36.577946 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:36.578424 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:03:36.578428 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:03:36.579717 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:03:36.581653 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:36.581654 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:36.581654 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:36.581858 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:36.582051 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:36.582052 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:36.582052 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:36.582128 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:36.582426 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:03:36.582426 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:03:36.582458 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 11:03:36.582523 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 11:03:36.582616 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260929 11:03:36.583051 17449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.575931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40124 (local address 127.26.120.252:37281)
0929 11:03:36.575967 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:36.575970 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:36.575977 (+ 7us) server_negotiation.cc:407] Connection header received
0929 11:03:36.576018 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:36.576019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:36.576028 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:36.576090 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:36.576296 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.576846 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.577563 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.577813 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.577883 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:36.579798 (+ 1915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:36.579799 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:36.579799 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:36.579805 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:36.581359 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:36.581935 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:36.581936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:36.581936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:36.581993 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:36.582198 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:36.582199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:36.582200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:36.582334 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:36.582405 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:36.582572 (+ 167us) server_negotiation.cc:299] Negotiation successful
0929 11:03:36.582679 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":20}
I20260929 11:03:36.583634 17375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.575688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44582 (local address 127.26.120.253:38943)
0929 11:03:36.575719 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:36.575721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:36.575728 (+ 7us) server_negotiation.cc:407] Connection header received
0929 11:03:36.575828 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:36.575829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:36.575838 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:36.575899 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:36.576502 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.576965 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.577618 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.577810 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.577887 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:36.580517 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:36.580518 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:36.580519 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:36.580524 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:36.582256 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:36.582623 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:36.582624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:36.582625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:36.582689 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:36.582909 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:36.582910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:36.582910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:36.583039 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:36.583308 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:36.583449 (+ 141us) server_negotiation.cc:299] Negotiation successful
0929 11:03:36.583520 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260929 11:03:36.585042 17446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4b01f92261a492494d431a8e640d766: Bootstrap starting.
I20260929 11:03:36.585510 27107 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 11:03:36.585811 17446 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4b01f92261a492494d431a8e640d766: Neither blocks nor log segments found. Creating new log.
Sep 29 11:03:36 dist-test-slave-2lzq krb5kdc[17215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679816, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.254@KRBTEST.COM for kudu/127.26.120.252@KRBTEST.COM
I20260929 11:03:36.586226 17446 log.cc:824] T 00000000000000000000000000000000 P c4b01f92261a492494d431a8e640d766: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:36.587082 17446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4b01f92261a492494d431a8e640d766: No bootstrap required, opened a new log
I20260929 11:03:36.586479 17268 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "2043e40ac747409988a3a2a145b687d4"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40044
I20260929 11:03:36.589177 17446 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4b01f92261a492494d431a8e640d766 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2043e40ac747409988a3a2a145b687d4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 45803 } } peers { permanent_uuid: "f2fb7867ab0f4521a3de5eaf96b2f19f" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 38943 } } peers { permanent_uuid: "c4b01f92261a492494d431a8e640d766" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 37281 } }
I20260929 11:03:36.589305 17299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.581127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:37281 (local address 127.0.0.1:40138)
0929 11:03:36.581185 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:36.581196 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:36.581241 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:36.581501 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:36.581507 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:36.581685 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:03:36.581729 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:36.581732 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.583230 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.583231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:36.584014 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:36.584017 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.584579 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.584579 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:36.584627 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:36.585000 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:03:36.585005 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:03:36.586463 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:03:36.588153 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:36.588154 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:36.588155 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:36.588366 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:36.588624 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:36.588625 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:36.588626 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:36.588696 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:36.589007 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:03:36.589008 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:03:36.589050 (+ 42us) client_negotiation.cc:769] Sending connection context
0929 11:03:36.589117 (+ 67us) client_negotiation.cc:240] Negotiation successful
0929 11:03:36.589209 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260929 11:03:36.589325 17446 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4b01f92261a492494d431a8e640d766 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b01f92261a492494d431a8e640d766, State: Initialized, Role: FOLLOWER
I20260929 11:03:36.589340 17452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.581223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40138 (local address 127.26.120.252:37281)
0929 11:03:36.581356 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:36.581358 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:36.581364 (+ 6us) server_negotiation.cc:407] Connection header received
0929 11:03:36.581381 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:36.581382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:36.581414 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:36.581480 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:36.581817 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.583121 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.584114 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.584421 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.584484 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:36.586557 (+ 2073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:36.586558 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:36.586559 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:36.586565 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:36.588078 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:36.588455 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:36.588456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:36.588456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:36.588512 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:36.588770 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:36.588771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:36.588771 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:36.588933 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:36.589008 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:36.589163 (+ 155us) server_negotiation.cc:299] Negotiation successful
0929 11:03:36.589241 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":87,"threads_started":1}
I20260929 11:03:36.589419 17446 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4b01f92261a492494d431a8e640d766 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2043e40ac747409988a3a2a145b687d4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 45803 } } peers { permanent_uuid: "f2fb7867ab0f4521a3de5eaf96b2f19f" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 38943 } } peers { permanent_uuid: "c4b01f92261a492494d431a8e640d766" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 37281 } }
I20260929 11:03:36.590061 17446 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4b01f92261a492494d431a8e640d766 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:03:36.590186 17454 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4b01f92261a492494d431a8e640d766 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2043e40ac747409988a3a2a145b687d4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 45803 } } peers { permanent_uuid: "f2fb7867ab0f4521a3de5eaf96b2f19f" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 38943 } } peers { permanent_uuid: "c4b01f92261a492494d431a8e640d766" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 37281 } } }
I20260929 11:03:36.590274 17454 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4b01f92261a492494d431a8e640d766 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 11:03:36.590939 17296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2043e40ac747409988a3a2a145b687d4: Bootstrap starting.
I20260929 11:03:36.591686 17296 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2043e40ac747409988a3a2a145b687d4: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:36.591996 17296 log.cc:824] T 00000000000000000000000000000000 P 2043e40ac747409988a3a2a145b687d4: Log is configured to *not* fsync() on all Append() calls
W20260929 11:03:36.592192 17465 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c4b01f92261a492494d431a8e640d766: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 11:03:36.592299 17465 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 11:03:36.592757 17296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2043e40ac747409988a3a2a145b687d4: No bootstrap required, opened a new log
I20260929 11:03:36.594110 17296 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2043e40ac747409988a3a2a145b687d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2043e40ac747409988a3a2a145b687d4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 45803 } } peers { permanent_uuid: "f2fb7867ab0f4521a3de5eaf96b2f19f" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 38943 } } peers { permanent_uuid: "c4b01f92261a492494d431a8e640d766" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 37281 } }
I20260929 11:03:36.594245 17296 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2043e40ac747409988a3a2a145b687d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2043e40ac747409988a3a2a145b687d4, State: Initialized, Role: FOLLOWER
I20260929 11:03:36.594337 17296 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2043e40ac747409988a3a2a145b687d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2043e40ac747409988a3a2a145b687d4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 45803 } } peers { permanent_uuid: "f2fb7867ab0f4521a3de5eaf96b2f19f" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 38943 } } peers { permanent_uuid: "c4b01f92261a492494d431a8e640d766" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 37281 } }
I20260929 11:03:36.592202 17378 master_runner.cc:186] Error getting master registration for 127.26.120.254:45803: OK, instance_id { permanent_uuid: "2043e40ac747409988a3a2a145b687d4" instance_seqno: 1790679816234502 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 11:03:36.594521 17378 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 11:03:36.594605 17296 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2043e40ac747409988a3a2a145b687d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:03:36.594631 17268 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2043e40ac747409988a3a2a145b687d4 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260929 11:03:36.594707 17268 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2043e40ac747409988a3a2a145b687d4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:36.595265 17268 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2043e40ac747409988a3a2a145b687d4 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2043e40ac747409988a3a2a145b687d4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 45803 } } peers { permanent_uuid: "f2fb7867ab0f4521a3de5eaf96b2f19f" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 38943 } } peers { permanent_uuid: "c4b01f92261a492494d431a8e640d766" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 37281 } }
I20260929 11:03:36.595472 17268 leader_election.cc:288] T 00000000000000000000000000000000 P 2043e40ac747409988a3a2a145b687d4 [CANDIDATE]: Term 1 election: Requested vote from peers f2fb7867ab0f4521a3de5eaf96b2f19f (127.26.120.253:38943), c4b01f92261a492494d431a8e640d766 (127.26.120.252:37281)
I20260929 11:03:36.595757 17420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2043e40ac747409988a3a2a145b687d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c4b01f92261a492494d431a8e640d766"
I20260929 11:03:36.595849 17420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4b01f92261a492494d431a8e640d766 [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:36.596629 17420 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c4b01f92261a492494d431a8e640d766 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2043e40ac747409988a3a2a145b687d4 in term 1.
I20260929 11:03:36.596859 17247 leader_election.cc:302] T 00000000000000000000000000000000 P 2043e40ac747409988a3a2a145b687d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2043e40ac747409988a3a2a145b687d4, c4b01f92261a492494d431a8e640d766; no voters:
W20260929 11:03:36.597234 17483 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2043e40ac747409988a3a2a145b687d4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 11:03:36.597371 17483 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 11:03:36.597831 17228 master_runner.cc:186] Error getting master registration for 127.26.120.253:38943: OK, instance_id { permanent_uuid: "f2fb7867ab0f4521a3de5eaf96b2f19f" instance_seqno: 1790679816394749 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 11:03:36.600214 17228 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 11:03:36.606359 17468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2043e40ac747409988a3a2a145b687d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2043e40ac747409988a3a2a145b687d4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 45803 } } peers { permanent_uuid: "f2fb7867ab0f4521a3de5eaf96b2f19f" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 38943 } } peers { permanent_uuid: "c4b01f92261a492494d431a8e640d766" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 37281 } } }
I20260929 11:03:36.606457 17468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2043e40ac747409988a3a2a145b687d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 11:03:36.606508 17468 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2043e40ac747409988a3a2a145b687d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 11:03:36.606588 17468 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2043e40ac747409988a3a2a145b687d4 [term 1 LEADER]: Becoming Leader. State: Replica: 2043e40ac747409988a3a2a145b687d4, State: Running, Role: LEADER
I20260929 11:03:36.606664 17468 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2043e40ac747409988a3a2a145b687d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2043e40ac747409988a3a2a145b687d4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 45803 } } peers { permanent_uuid: "f2fb7867ab0f4521a3de5eaf96b2f19f" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 38943 } } peers { permanent_uuid: "c4b01f92261a492494d431a8e640d766" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 37281 } }
I20260929 11:03:36.607095 17468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2043e40ac747409988a3a2a145b687d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2043e40ac747409988a3a2a145b687d4. Latest consensus state: current_term: 1 leader_uuid: "2043e40ac747409988a3a2a145b687d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2043e40ac747409988a3a2a145b687d4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 45803 } } peers { permanent_uuid: "f2fb7867ab0f4521a3de5eaf96b2f19f" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 38943 } } peers { permanent_uuid: "c4b01f92261a492494d431a8e640d766" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 37281 } } }
I20260929 11:03:36.607244 17468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2043e40ac747409988a3a2a145b687d4 [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:36.607501 17490 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 11:03:36.607705 17490 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 11:03:36.608270 17375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.600454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44590 (local address 127.26.120.253:38943)
0929 11:03:36.600491 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:36.600493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:36.600501 (+ 8us) server_negotiation.cc:407] Connection header received
0929 11:03:36.600522 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:36.600523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:36.600532 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:36.600584 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:36.601296 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.601962 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.602957 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.603703 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.603775 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:36.605259 (+ 1484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:36.605260 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:36.605261 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:36.605267 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:36.607022 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:36.607429 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:36.607430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:36.607431 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:36.607487 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:36.607710 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:36.607711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:36.607712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:36.607838 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:36.607909 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:36.608109 (+ 200us) server_negotiation.cc:299] Negotiation successful
0929 11:03:36.608182 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 11:03:36.609961 17420 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c4b01f92261a492494d431a8e640d766 [term 1 FOLLOWER]: Refusing update from remote peer 2043e40ac747409988a3a2a145b687d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 11:03:36.610306 17484 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2043e40ac747409988a3a2a145b687d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b01f92261a492494d431a8e640d766" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 37281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 11:03:36.613350 17454 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4b01f92261a492494d431a8e640d766 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2043e40ac747409988a3a2a145b687d4. Latest consensus state: current_term: 1 leader_uuid: "2043e40ac747409988a3a2a145b687d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2043e40ac747409988a3a2a145b687d4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 45803 } } peers { permanent_uuid: "f2fb7867ab0f4521a3de5eaf96b2f19f" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 38943 } } peers { permanent_uuid: "c4b01f92261a492494d431a8e640d766" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 37281 } } }
I20260929 11:03:36.613459 17454 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4b01f92261a492494d431a8e640d766 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 11:03:36.614790 17452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.597676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40140 (local address 127.26.120.252:37281)
0929 11:03:36.597711 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:36.597713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:36.598169 (+ 456us) server_negotiation.cc:407] Connection header received
0929 11:03:36.598318 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:36.598319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:36.598333 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:36.598374 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:36.599531 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.600235 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.601868 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.602237 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.602280 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:36.611011 (+ 8731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:36.611012 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:36.611013 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:36.611020 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:36.612620 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:36.613852 (+ 1232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:36.613853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:36.613853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:36.613907 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:36.614157 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:36.614158 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:36.614158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:36.614320 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:36.614413 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:36.614577 (+ 164us) server_negotiation.cc:299] Negotiation successful
0929 11:03:36.614657 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 11:03:36.617410 17484 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2043e40ac747409988a3a2a145b687d4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2043e40ac747409988a3a2a145b687d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2043e40ac747409988a3a2a145b687d4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 45803 } } peers { permanent_uuid: "f2fb7867ab0f4521a3de5eaf96b2f19f" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 38943 } } peers { permanent_uuid: "c4b01f92261a492494d431a8e640d766" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 37281 } } }
I20260929 11:03:36.617509 17484 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2043e40ac747409988a3a2a145b687d4 [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:36.617990 17454 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4b01f92261a492494d431a8e640d766 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2043e40ac747409988a3a2a145b687d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2043e40ac747409988a3a2a145b687d4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 45803 } } peers { permanent_uuid: "f2fb7867ab0f4521a3de5eaf96b2f19f" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 38943 } } peers { permanent_uuid: "c4b01f92261a492494d431a8e640d766" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 37281 } } }
I20260929 11:03:36.618103 17454 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4b01f92261a492494d431a8e640d766 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 11:03:36.618134 17491 mvcc.cc:205] Tried to move back new op lower bound from 7334624528832131072 to 7334624528821702656. Current Snapshot: MvccSnapshot[applied={T|T < 7334624528832131072}]
I20260929 11:03:36.618281 17492 mvcc.cc:205] Tried to move back new op lower bound from 7334624528832131072 to 7334624528821702656. Current Snapshot: MvccSnapshot[applied={T|T < 7334624528832131072}]
I20260929 11:03:36.619693 17490 catalog_manager.cc:1347] Generated new cluster ID: 0ebc5a638e0f4ec0855a165e2c7f86d2
I20260929 11:03:36.619746 17490 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 11:03:36.624876 17451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.600344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40052 (local address 127.26.120.254:45803)
0929 11:03:36.600414 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:36.600416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:36.606341 (+ 5925us) server_negotiation.cc:407] Connection header received
0929 11:03:36.606368 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:36.606369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:36.606381 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:36.606440 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:36.607733 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.608445 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.609704 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.609979 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.610034 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:36.615957 (+ 5923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:36.615958 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:36.615959 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:36.615967 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:36.618008 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:36.618804 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:36.618805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:36.618806 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:36.618885 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:36.619080 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:36.619081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:36.619081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:36.619221 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:36.619319 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:36.619574 (+ 255us) server_negotiation.cc:299] Negotiation successful
0929 11:03:36.619680 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 11:03:36.626369 17490 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 11:03:36.627110 17490 catalog_manager.cc:1504] Loading token signing keys...
I20260929 11:03:36.631383 17371 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.120.252" port: 37281 } attempt: 3
Sep 29 11:03:36 dist-test-slave-2lzq krb5kdc[17215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790679816, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.120.253@KRBTEST.COM for kudu/127.26.120.252@KRBTEST.COM
I20260929 11:03:36.636438 17490 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2043e40ac747409988a3a2a145b687d4: Generated new TSK 0
I20260929 11:03:36.636626 17490 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 11:03:36.639377 17373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.631633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.120.252:37281 (local address 127.0.0.1:40144)
0929 11:03:36.631676 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0929 11:03:36.631687 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 11:03:36.631722 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 11:03:36.631875 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 11:03:36.631876 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 11:03:36.632055 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0929 11:03:36.632106 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:36.632108 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.632971 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.632972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:36.633798 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 11:03:36.633801 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.634290 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.634291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 11:03:36.634353 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:36.634813 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 11:03:36.634818 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 11:03:36.636276 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 11:03:36.638323 (+ 2047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:36.638324 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:36.638325 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:36.638497 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:36.638734 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 11:03:36.638735 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 11:03:36.638735 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 11:03:36.638790 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 11:03:36.639080 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 11:03:36.639081 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 11:03:36.639122 (+ 41us) client_negotiation.cc:769] Sending connection context
0929 11:03:36.639187 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 11:03:36.639289 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260929 11:03:36.639962 17452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:36.631706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40144 (local address 127.26.120.252:37281)
0929 11:03:36.631755 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:36.631757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:36.631765 (+ 8us) server_negotiation.cc:407] Connection header received
0929 11:03:36.631799 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:36.631799 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:36.631809 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:36.631858 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:36.632191 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.632906 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.633894 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:36.634228 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:36.634290 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:36.636350 (+ 2060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 11:03:36.636351 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 11:03:36.636351 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 11:03:36.636358 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 11:03:36.638236 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:36.638591 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:36.638593 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:36.638593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:36.638651 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 11:03:36.638861 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 11:03:36.638862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 11:03:36.638863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 11:03:36.638993 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 11:03:36.639052 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:36.639789 (+ 737us) server_negotiation.cc:299] Negotiation successful
0929 11:03:36.639869 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260929 11:03:36.641535 17371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2fb7867ab0f4521a3de5eaf96b2f19f: Bootstrap starting.
I20260929 11:03:36.642285 17371 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2fb7867ab0f4521a3de5eaf96b2f19f: Neither blocks nor log segments found. Creating new log.
I20260929 11:03:36.642562 17371 log.cc:824] T 00000000000000000000000000000000 P f2fb7867ab0f4521a3de5eaf96b2f19f: Log is configured to *not* fsync() on all Append() calls
I20260929 11:03:36.643510 17371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2fb7867ab0f4521a3de5eaf96b2f19f: No bootstrap required, opened a new log
I20260929 11:03:36.645869 17371 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2fb7867ab0f4521a3de5eaf96b2f19f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2043e40ac747409988a3a2a145b687d4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 45803 } } peers { permanent_uuid: "f2fb7867ab0f4521a3de5eaf96b2f19f" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 38943 } } peers { permanent_uuid: "c4b01f92261a492494d431a8e640d766" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 37281 } }
I20260929 11:03:36.646010 17371 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2fb7867ab0f4521a3de5eaf96b2f19f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2fb7867ab0f4521a3de5eaf96b2f19f, State: Initialized, Role: FOLLOWER
I20260929 11:03:36.646139 17371 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2fb7867ab0f4521a3de5eaf96b2f19f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2043e40ac747409988a3a2a145b687d4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 45803 } } peers { permanent_uuid: "f2fb7867ab0f4521a3de5eaf96b2f19f" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 38943 } } peers { permanent_uuid: "c4b01f92261a492494d431a8e640d766" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 37281 } }
I20260929 11:03:36.646591 17371 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2fb7867ab0f4521a3de5eaf96b2f19f [sys.catalog]: configured and running, proceeding with master startup.
I20260929 11:03:36.646628 17345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2043e40ac747409988a3a2a145b687d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f2fb7867ab0f4521a3de5eaf96b2f19f"
I20260929 11:03:36.646725 17345 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2fb7867ab0f4521a3de5eaf96b2f19f [term 0 FOLLOWER]: Advancing to term 1
I20260929 11:03:36.647615 17345 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f2fb7867ab0f4521a3de5eaf96b2f19f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2043e40ac747409988a3a2a145b687d4 in term 1.
I20260929 11:03:36.648038 17344 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f2fb7867ab0f4521a3de5eaf96b2f19f [term 1 FOLLOWER]: Refusing update from remote peer 2043e40ac747409988a3a2a145b687d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260929 11:03:36.650018 17511 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f2fb7867ab0f4521a3de5eaf96b2f19f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 11:03:36.650091 17511 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 11:03:36.650382 17500 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2fb7867ab0f4521a3de5eaf96b2f19f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2043e40ac747409988a3a2a145b687d4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 45803 } } peers { permanent_uuid: "f2fb7867ab0f4521a3de5eaf96b2f19f" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 38943 } } peers { permanent_uuid: "c4b01f92261a492494d431a8e640d766" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 37281 } } }
I20260929 11:03:36.650473 17500 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2fb7867ab0f4521a3de5eaf96b2f19f [sys.catalog]: This master's current role is: FOLLOWER
I20260929 11:03:36.650915 17497 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2043e40ac747409988a3a2a145b687d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2fb7867ab0f4521a3de5eaf96b2f19f" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 11:03:36.652393 17515 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2fb7867ab0f4521a3de5eaf96b2f19f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2043e40ac747409988a3a2a145b687d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2043e40ac747409988a3a2a145b687d4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 45803 } } peers { permanent_uuid: "f2fb7867ab0f4521a3de5eaf96b2f19f" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 38943 } } peers { permanent_uuid: "c4b01f92261a492494d431a8e640d766" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 37281 } } }
I20260929 11:03:36.652420 17500 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2fb7867ab0f4521a3de5eaf96b2f19f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2043e40ac747409988a3a2a145b687d4. Latest consensus state: current_term: 1 leader_uuid: "2043e40ac747409988a3a2a145b687d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2043e40ac747409988a3a2a145b687d4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 45803 } } peers { permanent_uuid: "f2fb7867ab0f4521a3de5eaf96b2f19f" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 38943 } } peers { permanent_uuid: "c4b01f92261a492494d431a8e640d766" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 37281 } } }
I20260929 11:03:36.652542 17515 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2fb7867ab0f4521a3de5eaf96b2f19f [sys.catalog]: This master's current role is: FOLLOWER
I20260929 11:03:36.652556 17500 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2fb7867ab0f4521a3de5eaf96b2f19f [sys.catalog]: This master's current role is: FOLLOWER
I20260929 11:03:36.653898 17513 mvcc.cc:205] Tried to move back new op lower bound from 7334624528936955904 to 7334624528821702656. Current Snapshot: MvccSnapshot[applied={T|T < 7334624528832131072}]
W20260929 11:03:36.670146 17452 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 11:03:36.668780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40156 (local address 127.26.120.252:37281)
0929 11:03:36.668826 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:36.668828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:36.669462 (+ 634us) server_negotiation.cc:407] Connection header received
0929 11:03:36.669570 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:36.669571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:36.669582 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:36.669613 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:36.670075 (+ 462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40156: BlockingRecv error: recv got EOF from 127.0.0.1:40156 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260929 11:03:36.670925 17451 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 11:03:36.669179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40056 (local address 127.26.120.254:45803)
0929 11:03:36.669219 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:36.669221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:36.670389 (+ 1168us) server_negotiation.cc:407] Connection header received
0929 11:03:36.670484 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:36.670485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:36.670496 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:36.670524 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:36.670866 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40056: BlockingRecv error: recv got EOF from 127.0.0.1:40056 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260929 11:03:36.671918 17375 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 11:03:36.671236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44594 (local address 127.26.120.253:38943)
0929 11:03:36.671284 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:36.671286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:36.671408 (+ 122us) server_negotiation.cc:407] Connection header received
0929 11:03:36.671500 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:36.671501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:36.671511 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:36.671542 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:36.671866 (+ 324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44594: BlockingRecv error: recv got EOF from 127.0.0.1:44594 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260929 11:03:36.675014 17375 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 11:03:36.674290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44610 (local address 127.26.120.253:38943)
0929 11:03:36.674323 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:36.674324 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:36.674537 (+ 213us) server_negotiation.cc:407] Connection header received
0929 11:03:36.674636 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:36.674637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:36.674653 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:36.674681 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:36.674970 (+ 289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44610: BlockingRecv error: recv got EOF from 127.0.0.1:44610 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260929 11:03:36.677811 17375 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 11:03:36.677133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44622 (local address 127.26.120.253:38943)
0929 11:03:36.677164 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:36.677166 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:36.677368 (+ 202us) server_negotiation.cc:407] Connection header received
0929 11:03:36.677459 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:36.677459 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:36.677468 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:36.677495 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:36.677773 (+ 278us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44622: BlockingRecv error: recv got EOF from 127.0.0.1:44622 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260929 11:03:36.681502 17375 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 11:03:36.680834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44630 (local address 127.26.120.253:38943)
0929 11:03:36.680864 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:36.680865 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:36.681048 (+ 183us) server_negotiation.cc:407] Connection header received
0929 11:03:36.681136 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:36.681137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:36.681146 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:36.681174 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:36.681463 (+ 289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44630: BlockingRecv error: recv got EOF from 127.0.0.1:44630 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260929 11:03:36.687477 17375 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 11:03:36.686772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44636 (local address 127.26.120.253:38943)
0929 11:03:36.686803 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:36.686805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:36.687002 (+ 197us) server_negotiation.cc:407] Connection header received
0929 11:03:36.687092 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:36.687092 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:36.687101 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:36.687128 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:36.687435 (+ 307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44636: BlockingRecv error: recv got EOF from 127.0.0.1:44636 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260929 11:03:36.707449 17375 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 11:03:36.706445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44650 (local address 127.26.120.253:38943)
0929 11:03:36.706493 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:36.706496 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:36.706795 (+ 299us) server_negotiation.cc:407] Connection header received
0929 11:03:36.706910 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:36.706911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:36.706924 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:36.706957 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:36.707377 (+ 420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44650: BlockingRecv error: recv got EOF from 127.0.0.1:44650 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260929 11:03:36.725829 17375 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 11:03:36.724879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44664 (local address 127.26.120.253:38943)
0929 11:03:36.724925 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:36.724927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:36.725192 (+ 265us) server_negotiation.cc:407] Connection header received
0929 11:03:36.725321 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:36.725323 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:36.725335 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:36.725366 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:36.725763 (+ 397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44664: BlockingRecv error: recv got EOF from 127.0.0.1:44664 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260929 11:03:36.760092 17375 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 11:03:36.759253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44680 (local address 127.26.120.253:38943)
0929 11:03:36.759311 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:36.759314 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:36.759545 (+ 231us) server_negotiation.cc:407] Connection header received
0929 11:03:36.759645 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:36.759646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:36.759657 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:36.759687 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:36.760032 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44680: BlockingRecv error: recv got EOF from 127.0.0.1:44680 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260929 11:03:36.826215 17375 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 11:03:36.825454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44694 (local address 127.26.120.253:38943)
0929 11:03:36.825514 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:36.825517 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:36.825637 (+ 120us) server_negotiation.cc:407] Connection header received
0929 11:03:36.825690 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:36.825690 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:36.825700 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:36.825764 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:36.826142 (+ 378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44694: BlockingRecv error: recv got EOF from 127.0.0.1:44694 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260929 11:03:36.956034 17375 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 11:03:36.955399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44704 (local address 127.26.120.253:38943)
0929 11:03:36.955462 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:36.955465 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:36.955497 (+ 32us) server_negotiation.cc:407] Connection header received
0929 11:03:36.955567 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:36.955568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:36.955578 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:36.955640 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:36.955975 (+ 335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44704: BlockingRecv error: recv got EOF from 127.0.0.1:44704 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260929 11:03:37.214010 17375 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 11:03:37.213293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44712 (local address 127.26.120.253:38943)
0929 11:03:37.213360 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:37.213362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:37.213373 (+ 11us) server_negotiation.cc:407] Connection header received
0929 11:03:37.213452 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:37.213453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:37.213470 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:37.213537 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 11:03:37.213937 (+ 400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44712: BlockingRecv error: recv got EOF from 127.0.0.1:44712 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 11:03:37.254715 17497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2043e40ac747409988a3a2a145b687d4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2043e40ac747409988a3a2a145b687d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2043e40ac747409988a3a2a145b687d4" member_type: VOTER last_known_addr { host: "127.26.120.254" port: 45803 } } peers { permanent_uuid: "f2fb7867ab0f4521a3de5eaf96b2f19f" member_type: VOTER last_known_addr { host: "127.26.120.253" port: 38943 } } peers { permanent_uuid: "c4b01f92261a492494d431a8e640d766" member_type: VOTER last_known_addr { host: "127.26.120.252" port: 37281 } } }
I20260929 11:03:37.254869 17497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2043e40ac747409988a3a2a145b687d4 [sys.catalog]: This master's current role is: LEADER
I20260929 11:03:37.592764 17465 catalog_manager.cc:1259] Loaded cluster ID: 0ebc5a638e0f4ec0855a165e2c7f86d2
I20260929 11:03:37.592855 17465 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c4b01f92261a492494d431a8e640d766: loading cluster ID for follower catalog manager: success
I20260929 11:03:37.593669 17465 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c4b01f92261a492494d431a8e640d766: acquiring CA information for follower catalog manager: success
I20260929 11:03:37.593982 17465 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c4b01f92261a492494d431a8e640d766: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 11:03:37.650481 17511 catalog_manager.cc:1259] Loaded cluster ID: 0ebc5a638e0f4ec0855a165e2c7f86d2
I20260929 11:03:37.650550 17511 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f2fb7867ab0f4521a3de5eaf96b2f19f: loading cluster ID for follower catalog manager: success
I20260929 11:03:37.651355 17511 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f2fb7867ab0f4521a3de5eaf96b2f19f: acquiring CA information for follower catalog manager: success
I20260929 11:03:37.651670 17511 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f2fb7867ab0f4521a3de5eaf96b2f19f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 11:03:37.729812 17583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:37.727115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44718 (local address 127.26.120.253:38943)
0929 11:03:37.727239 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:37.727242 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:37.727295 (+ 53us) server_negotiation.cc:407] Connection header received
0929 11:03:37.727374 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:37.727375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:37.727450 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:37.727538 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 11:03:37.727692 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:37.728200 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:37.728976 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:37.729203 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:37.729278 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:37.729457 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 11:03:37.729575 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 11:03:37.729671 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:37.729695 (+ 24us) server_negotiation.cc:299] Negotiation successful
0929 11:03:37.729716 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":63,"threads_started":1}
I20260929 11:03:37.734009 17589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:37.731683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40166 (local address 127.26.120.252:37281)
0929 11:03:37.731788 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:37.731790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:37.731807 (+ 17us) server_negotiation.cc:407] Connection header received
0929 11:03:37.731881 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:37.731881 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:37.731915 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:37.731978 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 11:03:37.732089 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:37.732588 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:37.733358 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:37.733554 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:37.733603 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:37.733765 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 11:03:37.733865 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 11:03:37.733920 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:37.733934 (+ 14us) server_negotiation.cc:299] Negotiation successful
0929 11:03:37.733949 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":62,"threads_started":1}
I20260929 11:03:37.738823 17583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:37.735733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44732 (local address 127.26.120.253:38943)
0929 11:03:37.735770 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:37.735772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:37.735872 (+ 100us) server_negotiation.cc:407] Connection header received
0929 11:03:37.735972 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:37.735973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:37.735984 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:37.736039 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 11:03:37.736168 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:37.736908 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:37.737996 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:37.738305 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:37.738367 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:37.738558 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 11:03:37.738590 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 11:03:37.738638 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:37.738732 (+ 94us) server_negotiation.cc:299] Negotiation successful
0929 11:03:37.738754 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 11:03:37.739251 17596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:37.735740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40062 (local address 127.26.120.254:45803)
0929 11:03:37.736017 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:37.736019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:37.736026 (+ 7us) server_negotiation.cc:407] Connection header received
0929 11:03:37.736067 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:37.736068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:37.736101 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:37.736223 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 11:03:37.736306 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:37.737034 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:37.738024 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:37.738300 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:37.738378 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:37.738560 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 11:03:37.738666 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 11:03:37.738726 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:37.739128 (+ 402us) server_negotiation.cc:299] Negotiation successful
0929 11:03:37.739155 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":54,"threads_started":1}
I20260929 11:03:37.739441 17589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 11:03:37.735708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40176 (local address 127.26.120.252:37281)
0929 11:03:37.735744 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0929 11:03:37.735746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 11:03:37.735921 (+ 175us) server_negotiation.cc:407] Connection header received
0929 11:03:37.736085 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 11:03:37.736086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 11:03:37.736095 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 11:03:37.736147 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 11:03:37.737165 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:37.737593 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:37.738621 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 11:03:37.738900 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 11:03:37.738966 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 11:03:37.739144 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 11:03:37.739171 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 11:03:37.739214 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0929 11:03:37.739272 (+ 58us) server_negotiation.cc:299] Negotiation successful
0929 11:03:37.739292 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 11:03:37.741209 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 17228
I20260929 11:03:37.746399 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 17303
I20260929 11:03:37.751387 27107 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkJyhTP/build/release/bin/kudu with pid 17378
2026-09-29T11:03:37Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2781 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5583 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (267770 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST